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report  describee  en  eveluetlon  of  the  utility  end 


perceived  ef fectiveneea  of  e  specific  structured  analysis  end  design  technique 
as  an  aid  in  managing  a  complex  military  training  environment.  In  this  study 
SofTech,  Inc. ,  a  Boston-based  computer  software  firm,  applied  their  Structured 
Analysis  and  Design  Technique  (sai$£®  )_  to  the  problems  confronting  the 


to  the  problems  confronting  the 


U.S.  Army  Tnlnlng  and  Doctrine  Command  (TRADOC)  in  managing  rapidly  occurring 
innovations  in  training. 

These  changes  include  the  self-pacing  of  a  large  number  of  training 
courses,  evaluating  training  effectiveness  both  in  terms  of  the  individual 
soldier  and  the  unit,  and  the  development  of  management  strategies  to  Improve 
TRADOC' s  ability  to  allocate  resources,  establish  accountability,  improve 
teaching  quality,  and  to  achieve  a  better  match  between  training  content  and 
job  objectives. 

A- 

If  use  of  this  particular  analysis  and  planning  methodology  signif¬ 
icantly  Improved  TRADOC' s  ability  to  manage  proposed  changes  in  training,  more 
widespread  use  of  SADT  in  the  DoD  training  environment  would  be  warranted. 

Thus,  the  project  included  an  independent  evaluation  commissioned  by  the 
Advanced  Research  Projects  Agency  (ARPA)  and  conducted  by  The  Instructional 
Systems  Laboratory  at  The  University  of  Texas  at  Austin.  The  evaluation 
staff  was  responsible  solely  to  ARPA  for  evaluative  procedures,  results, 
and  conclusions.  The  goals  of  this  evaluation  were  to  assess  the  utility 
and  effectiveness  of  SADT  in  a  training  environment.  This  document  constitutes 
the  final  report  of  this  evaluation. 


Overview  of  SADT 

SADT  analyses  a  problem  through  building  a  model  or  representation  of 
the  problem  on  paper  which  is  top-down,  hierarchic,  modular,  and  structured. 


SADT  consists  of  two  major  components,  graphic  notation  and  s  disciplined 
process  which  specifies  how  Individuals  in  clearly  defined  roles  are  to 
interact  to  produce  a  model  of  the  training  program.  The  benefits  of 
applying  this  technique  to  a  complex  system  problem  derive  from  the  model 
produced.  This  model  serves  as  documentation  of  the  best  thinking  of  experts 
as  to  the  status  of  the  system  and  evolves  through  an  iterative  author- 
commenter  cycle. 

Project  Tasks  and  Evaluation  Phases 

SofTech  was  to  perform  their  work  Ir  4  tasks.  Task  1  was  a  "start  up" 
activity  in  which  SofTech  analysts  produced  an  overview  model  of  the  training 
and  evaluation  system.  In  Task  2  SADT  analysts  from  SofTech  and  TRADOC 
jointly  produced  an  SADT  model  of  a  tank  weapons  system.  In  Task  3  TRADOC 
and  SADT  analysts  used  Tasks  1  and  2  models  as  starting  points  to  produce 
a  fully  elaborated  model  of  Army  training  and  evaluation  as  it  should  be 
conducted.  Task  4  was  to  consist  of  a  plan  for  implementing  the  training 
and  evaluation  system  conceptualised  in  Task  3,  but  was  subsequently  altered 
and  did  not  fall  within  the  timeframe  of  this  evaluation. 

The  evaluation  was  conducted  in  three  phases.  A  primary  source  of 
data  used  in  all  three  phases  was  the  expressed  opinions  and  attitudes  of 
participating  TRADOC  personnel  towards  SADT.  The  purpose  of  the  Phase  1 
evaluative  phase  was  to  determine  initial  attitudes  towards  SADT  and  to 
measure  the  kinds  of  activities  occurring  in  the  project.  The  Phase  2  evaluation 
reported  attitudes  towards  SADT  after  development  of  the  major  SADT  models 
were  completed.  The  Phase  3  evaluation  followed  Phase  2  by  6  weeks  and  was 
concerned  with  examining  discrepancies  between  Phase  1  and  Phase  2  results 
along  with  obtaining  suseuury  impressions  and  conclusions  about  SADT  from 
participating  TRADOC  personnel. 
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Results  and  Conclusion* 


Extensive  data  war*  collected  early  In  the  project  to  determine  If  the 
SADT  procues  had  been  implemented  as  specified  in  the  contract  proposal.  These 
data  suggested  that  the  participation  of  TRADOC  officers  in  the  project  had 
been  approximately  half  that  originally  expected.  The  number  of  officers 
trained  to  create  SADT  diagrams  was  less  than  expected  throughout  the  project 
and  the  failure  to  fully  implement  this  aspect  of  the  SADT  process  at  TRADOC 
may  have  lowered  the  Impact  the  technique  had  on  the  client  organisation. 

Participating  TRADOC  officers  were  polled  about  halfway  in  the  project 
concerning  their  impressions  of  SADT  and  the  progress  of  the  project  thus  far. 
The  general  opinion  expressed  was  that  while  the  technique  appeared  to  be  a 
good  one  in  a  general,  context-free  sense,  most  officers  believed  it  was  too 
early  to  conclude  whether  SADT  would  ultimately  affect  the  training  concepts 
held  by  TRADOC.  Some  officers  were  skeptical  of  this  project  having  any 
impact  on  TRADOC  or  the  status  of  training. 

Participating  TRADOC  officers  were  polled  again  following  the  completion 
of  Task  3.  It  was  found  from  comparing  these  two  samplings  that  attitudes 
towards  SADT  had  become  more  positive  following  the  completion  of  Task  3. 

At  this  point  participating  officers  had  gained  confidence  in  the  ability 
of  the  technique  to  focus  attention  on  relevant  issues  and  elucidate  the 
interrelationships  of  various  components  of  the  training  system.  These 
officers  were  particularly  pleased  with  the  Task  3  model,  which  appears  to 
have  documented  a  level  of  understanding  and  inright  into  the  training  system 
which  had  not  been  achieved  or  at  least  so  vividly  portrayed  previously.  A 
follow-up  interview  and  questionnaire  six  weeks  later  indicated  that  these 
expressed  opinions  remained  stable. 
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Summary  and  Conclusions 

Articulation  of  Array  training  and  evaluation  repreaented  by  the 
Task  3  model  represented  the  most  direct  tangible  benefit  of  this  project 
to  TRADOC.  This  nodel  integrated  the  previously  independently  conceptualised 
areas  of  training,  evaluation,  and  system  development  in  an  understandable 
and  practical  manner.  From  this  documentation,  the  actual  work  of  imple¬ 
menting  the  new  conceptualisation  of  training  can  begin. 

It  does  not  appear  that  this  application  of  SADT  created  new  know¬ 
ledge  about  specific  aspects  of  the  training  system  but  rather  that  it 
integrated  and  communicated  what  was  known  about  training  in  a  useful  and 
coherent  way.  The  SADT  diagrams  and  procedures  provide  a  mechanism  for 
documenting  the  best  thinking  in  a  client  organisation  and  interrelating 
important  concepts  in  a  clear  and  practical  manner.  It  was  concluded  that 
further  applications  of  SADT  such  as  the  one  described  in  this  report  would 
be  worthwhile. 
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I.  Project  Overview  * 


This  document  describes  the  evaluation  of  a  project  supported 
jointly  by  the  United  States  Army  Training  and  Doctrine  Crwmsnd  (TRADOC) 
and  the  Defense  Advanced  Research  Projects  Agency  (ASPA)  under  the 
contract  title  "Demonstration  and  Evaluation  of  an  Advanced  Systems  Analysis 
Technique  in  Modeling  «  DoD  Training  Environment"  (ARPA  Order  Ho,  3230, 
Contract  Ho.:  MDA  903-76-C-0249) .** 

In  this  study  SofTech,  Inc. ,  a  Boston-based  computer  software  firm, 

(r) 

applied  the  Structured  Analysis  and  Design  Technique  (SADT^)  to  the 
problems  confronting  a  training  command  headquarters  in  managing  rapidly 
occurring  Innovations  in  training  methods.  These  changes  include  the  self¬ 
pacing  of  a  large  number  of  Army  courses,  the  initiation  of  a  major  project 
to  evaluate  training  effectiveness  both  in  terms  of  the  individual  soldier 
and  the  unit,  and  the  development  of  management  strategies  to  improve  TRADOC’s 
ability  to  allocate  resources  to  training  sites,  establish  better  accounta¬ 
bility  of  resources  expended,  improve  the  management  of  teaching  quality, 
measure  competence  objectively  and  quantitatively,  and  to  achieve  a  better 
match  between  training  content  and  real  world  job  objectives. 

If  this  particular  analysis  and  planning  methodology  signflcantly 


*The  authors  would  like  to  thank  SofTech,  Inc.  for  their  cooperation 
and  assistance  in  gathering  data  for  this  report,  especially  Reuben  Jones, 
project  manager;  Steve  Llpka,  Stan  Smith  and  Boyd  Mathers,  SADT  authors;  as 
well  as  many  officers  at  TRADOC  who  freely  gave  of  their  valuable  time  for 

interviews  and  questionnaire  responding.  Without  the  cooperation  of  both 
agencies  this  report  could  not  have  been  written. 

♦♦Evaluation  reports  were  generated  throughout  this  project.  These 
interim  reports  were  written  for  a  limited  audience  and  were  not  dissmainated. 
Although  not  available,  these  reports  are  cited  in  the  following  text  to  pro¬ 
vide  background  and  clarity  to  ongoing  discussions.  The  substantive  results 
and  conclusions  of  these  sillier,  sequential  project  summaries  are  contained 
in  this  report. 
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lap roved  TRADOC'a  ability  to  accomplish  thee*  objectives ,  aore  wide-spread 
use  of  SADT  throughout  the  DoD  training  eetebllshaent ,  end  poeelbly  other 
perte  of  DoD  would  be  warranted.  Beceuee  of  the  rapid  pace  with  which 
training  innovations  are  planned  at  TRADOC,  this  environment  provided  a 
good  test  bed  In  which  to  aaaeea  the  applicability  of  SADT  to  analyala  and 
planning  problems  encountered  in  other  DoD  training  anvlronmenta. 

Thus,  this  project  Included  an  evaluation  effort  to  assess  the  utility 
and  effectiveness  of  the  Structured  Analysis  and  Design  Technique  (SADT) 
in  a  training  environment  and  an  evaluation  of  the  impact  of  the  project 
on  TRADOC' s  ability  to  identify  and  understand  the  types  of  changes  required 
in  Army  training  and  to  develop  a  practical  plan  for  carrying  out  those 
changes.  This  document  constitutes  the  final  report  of  this  evaluation. 
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II.  Overview  of  SADT 


SADT  le  *  systematic  approach  to  understanding  and  solving  complex 
system  problems,  Including  planning,  requirements  definition,  functional 
analysis,  and  system  design.  Under  development  by  ScfTech  since  1970,  the 
technique  grew  out  of  earlier  work  on  software  engineering  at  the  Massa¬ 
chusetts  Institute  of  Technology.  SofTech  has  applied  the  technique, 
to  a  wide  range  of  planning,  analysis,  and  design  probleaa,  Involving 
people,  aachlnes,  software,  hardware,  databases,  coaaunlcations  networks, 
procedures  and  finances. 

The  underlying  philosophy  of  SADT  la  that  the  human  mind  Is  capable 
of  understanding  any  amount  of  complexity,  as  long  as  it  is  presented  In 
small,  accessible  chunks  that  are  linked  together  to  make  a  whole.  This 
structured  decomposition  orientation  to  understanding  complex  probleaa  has 
become  a  standard  tool  of  computer  software  development  specialists  but 
would  appear  to  be  applicable  to  any  field  where  there  la  a  need  to  eluci¬ 
date  the  relationships  between  parts  and  wholes  in  complex  systems  or 
programs. 

SADT  analyses  a  problem  through  building  a  model  or  representation 
of  the  problem  on  paper  which  is  top-down,  modular,  hierarchic,  and 
structured.  SADT  consists  of  two  major  components,  the  graphic  notation 
of  this  particular  modeling  methodology  and  a  disciplined  process  which 
specifies  how  individuals  in  dearly  defined  roles  are  to  interact  to  produce 
the  model.  Thus,  SADT  consists  of  both  a  graphic  language  and  a  well- 
defined  discipline  or  process  which  spells  out  the  procedures  to  be  followed 
in  obtaining  a  structured  decomposition  of  the  problem  at  hand.  These  will 
be  discussed  in  turn  below. 


3 


Gtaphic Con vnt ions.  The  SADT  graphic  language  provldaa  a  llaltad 
set  of  constructs  froa  which  analysts  and  designers  can  coapose  orderly 
structures  of  any  required  site.  The  notation  laceaposed  of  boxes  and 
arrows.  Boxes  represent  parts  of  a  whole,  arrows  rupresent  Interfaces  between 
parts.  Diagrams  reprasent  wholes  and  are  coaq>osed  of  boxes,  arrows,  natural 
language  naaMs,  and  cartaln  other  notation.  The  aaae  graphics  are  applicable 
to  both  activities  and  data. 

An  SADT  model  is  an  organised  sequence  of  dlagraas,  each  with 
concise  supporting  text.  A  high-level  overview  diagram  represents  the  whole 
subject.  Each  lower-level  diagram  connects  exactly  Into  higher-level  portions 
of  the  model,  thus  preserving  tne  logical  relationship  of  each  component 
to  the  total  system.  Thus ,  program  detail  is  introduced  gradually  so  that 
substantive  detail  Is  Integrated  into  the  whole  without  obscuring  the 
overall  Intent  or  "big  picture".  Figure  1  provides  a  conceptual  illustration 
of  the  modeling  process.  Figure  2  contains  an  actual  SADT  diagram  produced 
during  this  project. 

An  SADT  model  is  a  graphic  representation  of  the  hierarchic  struc¬ 
ture  of  a  system,  decomposed  with  a  definite  purpose  in  mind.  A  model  is 
structured  so  that  it  gradually  exposes  detail,  but  its  depth  is  bounded 
by  the  restriction  of  Its  vantage  point;  and  its  content  is  bounded  by  Its 
viewpoint.  The  priorities  dictated  by  its  purpose  determine  the  layering 
of  the  top-down  decomposition.  Parallel  models  can  accomodate  both  multiple 
viewpoints  and  various  stages  of  system  implementation. 

The  arrow  structure  on  an  SADT  diagram  represents  a  constraint 
relationship  among  the  boxes.  It  does  not  necessarily  represent  flow  of 
control  or  sequence,  as  for  example,  on  a  flowchart  for  a  computer  program. 
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Figure  1.  Conceptual  illustration  of  the  modeling 
process . 


Constraint  arrows  show  necessary  conditions  imposed  on  an  activity. 

Most  arrows  represent  interfaces  between  boxes,  whether  in  the 
same  or  different  models.  Some  arrows  represent  non- interface  Interlocking 
between  models.  The  interface  structure,  particularly,  passes  through 
several  levels  of  diagrams,  creating  a  web  that  integrates  all  parts  of  the 
decomposition  and  shows  the  whole  system's  environmental  Interfaces  with 
the  topmost  box.  Further  documentation  of  the  mechanics  of  the  technique 
can  be  found  in  An  Introduction  to  SADT  (Publication  number  9022-78R, 
SofTech,  Inc.,  460  Totten  Pond  Road,  Waltham,  Massachusetts,  02154). 

The  SADT  Process.  Complex  problem  analysis  requires  cooperative 
teamwork  from  many  Individuals.  SADT  attempts  to  provide  a  clear  definition 
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diagram  produced  during  this  project. 


of  the  kinds  of  Interactions  which  should  occur  between  the  personnel 
Involved  by  establishing  titles  and  functions  of  appropriate  roles.  These 
appear  in  Table  1.  The  SADT  process,  in  which  these  roles  interact,  provides 
continuous  documentation,  and  regular  critical  review  of  the  work  produced. 

In  this  way  decisions  can  be  seen  in  context  and  can  be  challenged  while 
alternatives  are  still  viable. 

Throughout  a  project,  draft  versions  of  diagrams  in  evolving  models 
are  distributed  to  project  members  for  review.  Commeuters  make  their 
suggestions  in  writing  directly  on  copies  of  the  diagrams.  Written 
records  of  decisions  and  alternatives  are  retained  as  they  unfold.  As 
changes  and  corrections  are  made,  all  versions  are  entered  in  the  project 
files.  A  project  librarian  provides  filing,  distribution,  and  record  keeping 
support.  This  process  documents  all  decisions  and  the  reasons  why  decisions 
are  made.  When  commenters  and  authors  reach  an  understanding,  the  work  is 
reviewed  by  a  committee  of  senior  technical  and  management  personnel.  During 
the  process.  Incorrect  or  unacceptable  results  may  be  identified  early,  so 
that  oversights  or  errors  can  be  detected  before  they  cause  major  disruptions. 
Since  everything  is  on  record,  future  enhancement  and  system  maintenance 
can  reference  previous  analysis  and  design  decisions.  A  list  of  roles  and 
functions  used  in  the  SADT  process  appears  in  Table  1. 

Because  documentation  is  produced  as  the  model  evolves,  the 
status  of  the  project  is  visible  to  all  interested  parties.  Management  can 
study  the  requirements  (or  the  design)  in  a  "top-down"  manner,  beginning 
with  an  overview  and  continuing  to  any  relevant  level  of  detail.  Although 
presentations  to  upper  sanagement  usually  follow  standard  summary  and 
walk-through  methods,  even  senior  executives  can  become  readers  of  the 
SADT  language. 


Table  1 


Titles  and  Functions  In  the  SADT  Process 


Title 

Function 

Authors 

Personnel  who  study  requirements  and 
constraints,  analyze  the  aysten  functions 
and  represent  them  by  a  model  based  on 

SADT  diagrams. 

Commentera 

Individuals  who  must  review  and  comment 
in  writing  on  the  SADT  diagrams  produced 
by  the  authors. 

Technical  Committee 

A  group  of  senior  technical  personnel 
assigned  to  review  the  analysis  at 
every  major  level  of  decomposition.  They 
resolve  technical  questions  or  disagree¬ 
ments  or  recommend  a  decision  to  the 
project  management. 

Experts 

Persons  from  whom  authors  obtain  special¬ 
ized  information  about  functional  require¬ 
ments  and  certain  constraints  by  means 
of  interviews. 

Readers 

Personnel  who  read  SADT  diagrams  for  infor¬ 
mation  but  are  not  expected  to  make  written 
comments. 

Monitor 

A  person  fluent  in  SADT  who  assists  and 
advises  project  personnel  in  the  use  and 
application  of  SADT. 

Project  Librarian 

A  person  assigned  the  responsibility  of 
maintaining  a  centralized  file  of  all 
project  diagrams  and  associated  documents, 
making  copies,  distributing  reader  kits, 
keeping  reader  kits,  keeping  records,  etc. 

Project  Manager 

The  member  of  the  project  who  has  the 
final  technical  responsibility  for  carrying 
out  the  system  analysis  and  design. 

Instructor 

A  person  fluent  in  SADT  who  trains  Authors 
and  Commenters  who  are  using  SADT  for  the 
first  time. 
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Inplssmntlnx  th>  Approach.  How  SADI  is  implsmented  nriM  according 
to  organisation  oaeds  and  tha  kind a  of  tya terns  undar  conaldaratlon.  Thara 
ia  no  aat  pattern  aaong  different  organisations  for  the  contents  of  a 
problem  analysis.  In  each  caaa,  the  needs  of  the  client  organisation  euat 
be  accomodated.  Because  local  needs  are  diverse,  implementation  of  SADI  is 
a  "learn  by  doing”  experience  in  which  project  personnel  acquire  ways  of 
understanding  the  generic  nature  of  systems . 

While  little  previous  evaluative  data  exists  on  SADI,  users  report 
that  it  is  a  communications  vehicle  which  focuses  attention  on  well-defined 
topics,  that  it  increases  management  control  through  visibility  and  standard¬ 
isation,  that  It  creates  a  systematic  work  breakdown  structure  for  project 
teams,  and  that  it  minimises  errors  through  disciplined  flexibility  (see 
Ross,  D.T.  and  Schoman,  K.E.  Structured  analysis  for  requirements  definition. 
IEEE  Transactions  on  Software  Engineering,  Vol.  SR-3  (1),  January,  1977,  pp. 
6-15) . 
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III.  Overview  of  the  U.S.  Army  Training 
and  Doctrine  Co— nd  (TRADOC)* 

The  U.S.  Any  Training  and  Doctrine  Cnsmsnd  (TRADOC)  was  created 
in  1973.  With  an  annual  budgat  approaching  one  billion  dollara  and  a  total 
prograa  that  includea  the  employment  of  approximately  12,000  offlcera, 

50,000  enlisted  nan  and  40,000  civilians,  TRADOC  is  charged  with  the  responsi¬ 
bility  of  establishing  and  maintaining  Any  concepts,  principles,  and  policies 
(doctrine)  and  for  the  conduct  of  all  Any  training. 

TRADOC  consists  essentially  of  two  major  components  for  meeting 
these  responsibilities:  a  combat  developments  program  and  a  training  program. 
The  combat  developsmnts  program  consists  of  three  major  activities  (1)  the 
determination  of  requirements  and  capabilitlea  of  weapon  systems  and 
equipment  (Including  non-combat  equipment);  (2)  the  development  of  optimal 
organisational  policy  (e.g. ,  determining  the  formal  organisation  of  tank 
battalions  and  the  integration  of  those  forces  with  mechanised  rifle  com¬ 
panies,  maintenance  crews,  etc.);  and  (3)  the  development  of  tactics  and 
techniques  by  which  the  acquired  weapons  and  equipment  will  be  deployed 
by  the  various  organisational  groupings  of  troops  on  the  modern  battlefield. 

The  training  branch  o;T  TRADOC  consists  of  the  traditional  Army 
training  structure  described  below,  and  a  training  development  component 
which  has  achieved  visibility  only  within  the  last  two  years.  Each  will 
be  discussed  below. 


*Thls  description  of  TRADOC  is  based  on  the  proceedings  of  various 
Coamander's  Conferences  and  Technology  Symposiums  sponsored  by  TRADOC  in 
1975  and  1976.  While  this  overview  nay  now  be  slightly  dated,  it  illustrates 
considerations  which  ultimately  led  to  TRADOC' s  participation  in  this  project. 
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While  the  training  development*  component  of  TRADOC  repreeente 


*  new  concept,  the  training  structure  within  the  Artsy  is  well  established. 

For  enlisted  nen,  the  program  consist*  of  the  well-known  basic  training 
and  advanced  individual  training  (AIT)  which  ia  conducted  at  several 
Installations  across  the  country.  There  are  21  training  school*  such 
as  the  U.S.  Army  Infantry  School  at  Ft.  Banning,  Georgia,  the  U.S.  Army 
Armory  School  at  Ft.  Knox,  Kentucky,  the  U.S.  Army  Field  Artillery  School 
at  Ft.  Sill,  Oklahoma,  the  U.S.  Army  Transportation  School  at  Ft.  Bustle, 
Virginia,  and  the  U.S.  Army  Engineer  Training  Canter  at  Ft.  Leonard  Wood, 
•lissouri.  There  is  a  continuing  education  program  for  noncommissioned 
officers  (which,  like  AIT,  is  usually  conducted  through  the  training  school) 
to  keep  soldiers  currant  on  developments  in  equipment  and  doctrine. 

Thar*  la  also  basic  training,  intensive  advanced  individual 
training  and  continuing  education  programs  for  officers  which  contain 
3peclal  emphasis  on  such  areas  aa  leadership,  management,  and  tactics. 

Tor  higher  ranking  officers  (usually  majors)  there  is  the  Command  and 
General  Staff  College  at  Ft.  Levenworth,  Kansas,  and  at  a  still  higher 
level,  there  ia  the  Army  War  Collie.  Additionally,  the  entire  ROTC 
structure  and  Officor  Candidate  School  fall  within  TRADOC's  responsibilities. 

In  addition  to  these  training  facilities,  TRADOC  contains  many 
subagencies  such  as  the  Combat  Arms  Training  Board  (CATB) ,  the  Training 
Aid  Development  and  Requirements  Agency  (TRADER) ,  the  Combat  Development 
Experimentation  Coemsnd  (CDEC) ,  the  TRADOC  Combined  Arms  Test  Agency 
(TCATA)  and  the  TRADOC  Systems  Analysis  Agency  (TRASANA).  Thus,  manage¬ 
ment  responsibility  for  the  Various  aspects  of  training  and  evaluation 
;as  traditionally  been  diffused  throughout  the  training  system. 
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Mill*  this  brisf  survey  of  training  conands  and  agendas  is  not 
exhaustive.  It  indicates  the  breadth  of  TEADOC ' a  tnhiat  Iona  I  mamigcment 
responsibility.  Thus,  TEADOC  represents  perhaps  the  world's  orgeat 
training  agency ,  responsible  for  teaching  a  comprehensive  range  of  skills 
and  competencies  across  s  wide  range  of  geographic  locales  under  the 
control  of  ft  wide  range  of  subagencies  within  TEADOC. 

Innovations  in  Amy  Training 

Until  recently,  innovation  had  not  been  a  high  priority  in  Any 
training  and  evaluation.  Traditionally,  job  training  had  teen  conducted 
in  the  Any  schools  using  a  platform  lecture  format.  Performance  usually 
was  evaluated  by  paper  and  pencil  teats.  For  each  military  occupation 
specialty  (NOS)  there  was  an  AIT  course  and  progressive  supplementary 
training  courses.  There  was  an  associated  test  for  each  skill  level  of 
each  NOS,  usually  consisting  of  about  100  multiple  choice  questions. 

Initial  analyses  of  training  effectiveness  in  the  early  1970' s 
suggested  that  the  actual  performance  of  soldiers  In  the  field  did  not 
meet  crite- '  established  by  outside  contractors  and  the  combat  develop¬ 
ments  branch  of  TEADOC.  A  brief  example  illustrates  this  point.  While 
the  example  presented  is  hypothetical,  it  is  based  on  the  results  of 
several  actual analyses  of  specific  weapons  systems. 

Figure  3  Indicates  the  probability  of  hitting  a  moving  target 
with  a  particular  weapon  as  a  function  of  the  distance  from  the  target 
in  meters.  Line  A  indicates  the  actual  performance  of  a  group  of  soldiers 
using  this  weapon.  Line  B  Indicates  the  capability  of  the  weapon  as 
determined  by  the  Army  Materiel  Systems  Analysis  Agency  (AMSAA).  Line  C 
represents  what  a  group  of  combat  officers  believed  was  the  capability 
of  their  soldiers  of  using  the  weapon. 


13 


Figure  3.  Probability  of  hitting  a  moving  target  (Phlt> 
as  a  function  of  the  distance  fro«  the 
target  in  «eters. 


Figure  3  data  auggaat  that  tha  ability  of  the  soldier  to  ua«  tha 


weapon  in  tha  field  did  not  match  tha  weapon* a  capability  (discrepancy 
between  lines  A  and  B).  Further,  what  contending  officers  believed  to  be 
the  capability  of  the  weapon  waa  not  veridical  with  actual  performance 
(dlacrepancy  between  lines  A  and  C).  Thus,  the  decisions  an  officer  nsde 
on  thi  battlefield  relative  to  this  weapon  would  not  match  his  troops' 
ability  to  deploy  it  nor  the  weapon's  actual  capability. 

Based  on  these  results,  the  length  of  treining  for  this  weapon 
was  doubled  and  a  simulator  developed  to  provide  tore  practice  opportunities. 
This  increased  proficiency  considerably.  In  experimenting  with  the  training 
for  this  weapon  it  waa  found  that  soldiers  had  difficulty  estimating  the 
range  of  the  target  through  the  weapon's  complex  sight.  To  remedy  this  it 
was  decldec  to  use  the  weapons  in  pairs  and  firing  in  sequence.  Thus,  one 
soldier  would  estimate  renge,  aim,  and  fire.  A  second  soldier  would  then 
use  the  first  soldier's  range  estimate  and  his  observation  of  where  the 
first  round  hit  to  obtain  a  better  range  estimate  for  a  second  firing  on 
the  target.  The  performance  of  pairs  of  soldiers  using  thin  procedure 
is  Indicated  by  Line  0  on  Figure  2.  This  level  of  performence  exceeds 
what  waa  originally  considered  to  be  thu  capability  of  the  weapon.  Training 
analysis  studies  such  as  this  were  sesdnal  in  bringing  about  a  concern  for 
training  developments  within  TKADOC. 

Another  factor  motivating  the  search  for  improved  training  methods 
was  budgetary  constraints.  As  TKADOC  was  becoming  aware  of  soam  inadequacies 
in  treining,  they  were  feced  with  the  task  of  improving  the  quality  of 

e 

training  with  a  decreased  treining  budget.  This  forced  the  realisation  that 
the  traditional  labor  intensive  methods  of  training  employed  by  TKADOC 
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(platform  lacturars,  low  instructor /atudant  ratios,  ate.)  ware  not  a 
cost  officiant  approach  for  futura  Army  training.  Furtharmora,  moat 

soldiers  spand  only  2  or  3  months  out  of  a  thrae-yaar  tour  of  duty  at  a 
training  school.  The  man  hours  lostwhila  a  soldiar  is  away  from  his 
unit  of  assignment  attending  achool  and  the  associated  coats  contribute 
to  the  coat  of  traditional  Army  training.  These  considerations  hava 
led  TRADOC  to  make  training  available  to  the  soldiar  in  hia  unit  of 
assignment.  This  was  seen  aa  no  small  task  in  that  almost  all  training 

resources  ware  located  at  the  schools  and  training  centers. 

Given  these  cone lderat ions.  TRADOC  formed  an  instructional  tech¬ 
nology  study  group  whose  membership  included  representatives  of  Industry, 
government,  academia,  and  the  military.  After  studying  the  Army  training 
system  this  group  presented  its  findings  and  recoMendat ions  to  the  leader¬ 
ship  of  TRADOC.  While  theee  findings  are  too  lengthy  to  present  here 
several  key  Issues  that  emerged  are  summarised  below  to  convey  the  flavor 
of  what  was  reported  by  this  group. 

1.  The  Army's  instructional  strategies  were  not  as  cost- 
efficient  as  some  more  modem  instructional  technologies. 

2.  School  personnel  lacked  the  understanding,  motivation, 
and  skills  to  employ  modem  instructional  technologies. 

3.  Effective  training  program  evaluation  was  not  occurring 
in  the  Army  training  system. 

4.  Criterion-referenced  training  and  evaluation  was  not 
evident  in  the  Army  training  system. 

5.  Very  little  self-pacing  waa  being  employed  in  the  training 

system. 
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6.  Disincentives  for  the  Introduction  of  training 
innovations  abounded  in  the  training  ays tea. 

7.  There  was  no  proponeucy  within  TRADOC  devoted  to  the 
oanageoent  of  training  effactivsneas. 

8.  The  organisational  and  oanageoent  processes  of  TRADOC 
were  fractionalised  or  diffused  to  the  extent  that 
training  prograa  development,  implementation,  and 
evaluation  were  impeded. 

From  their  study,  this  consulting  group  recomended  that  TRADOC 
consolidate  their  training  resource  management  processes,  fora  a  training 
management  institute  to  orient  and  educate  the  various  managers  of  train¬ 
ing  in  modern  educational  technology,  consider  adopting  a  systems 
approach  to  training  based  on  job  performance  data  and  criterion-refer¬ 
enced  evaluation,  and  provide  incentives  to  school  and  unit  coamanders 
for  incorporating  advanced  instructional  techniques  and  establishing 
pilot  projects  towards  these  goals  as  soon  as  possible.* 

Since  its  inception  in  1973,  TRADOC  has  been  active  in  its 
attempt*  to  modernise  Army  training,  primarily  through  its  training 
developments  branch.  Major  projects  have  included  the  development  of 
packaged  training  extension  courses  (TECs)  which  can  be  sent  to  the 
units  thus  making  it  eaaier  for  the  soldier  to  learn  on  the  job.  Most 
of  these  courses  utilise  an  audio-visual  cassette  machine  that  presents 
a  lesson  both  verbally  and  through  visual  aids,  and  allows  for  tasting 
and  feedback  throughout  the  lesson.  Soms  machines  aven  allow  simulations 

*The  aforawantionad  activities  ars  not  intsnded  to  be  exhaustive 
of  *-he  influences  leading  to  an  increased  concern  for  the  status  of  training 
in  the  Aray  but  rather  exemplary  of  the  considerations  which  lad  to  change. 
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of  the  took  being  taught.  Evidence  Indicate*  that  the  self-paced  TEC 
coureea  are  at  leaat  as  effective  ae  traditional  Any  training  methods. 

At  the  and  of  FY  77,  10A4  of  the  5000  coureea  which  had  been  propoaed 
had  been  developed.  By  19  '9  the  Any  expect  a  to  have  5  million  copies 
of  the  TEC  lesaone  available  for  active  and  reserve  troops. 

Primarily  through  the  efforts  of  the  recently  formed  Training 
Management  Institute,  Skill  Qualification  Tests  (SQTs)  have  been  developed 
to  replace  the  old  pencil  and  paper  MOS  teats.  These  are  criterion- refer¬ 
enced  performance  tests  based  on  job  task  analysis  for  each  MOS.  TRADOC 
is  well  on  its  way  toward  developing  SQTs  for  each  skill  level  of  each 
MOS  and  towards  the  development  of  a  prototype  task  description  data  base. 

Another  major  effort  has  been  devoted  to  the  development  of 
training  and  evaluation  of  collectives  (groups) .  The  Army  Training  and 
Evaluation  Program  (ARTEP)  is  an  initial  effort  in  this  direction.  ARTEPs 
are  manuals  containing  "how  to"  instructions  for  unit  commanders  on 
conducting  collective  training  and  evaluation.  They  describe  the  cues 
and  conditions  under  which  a  particular  task  may  occur  as  part  of  some 
large  defined  mission  (e.g.,  "deliberate  daylight  attack"  or  "night  with¬ 
drawal"  of  a  rifle  company).  The  ARTEP  manual  also  specifies  the  training 
and  evaluation  standards  for  judging  the  performance  of  a  group. 

Another  major  TRADOC  effort  has  been  the  development  of  simulators 
to  better  train  soldiers  in  the  use  of  various  weapons  and  equipment.  One 
example  is  the  LASER  simulator  for  the  M16  rifle,  which  emits  an  eye-safe 
laser  beam  when  fired,  rather  than  a  live  round.  Sensors  on  targets 
provide  iaaediate  feedback  on  performance.  It  has  been  found  that  training 
with  LASER  simulators  is  at  least  as  effective  as  traditional  training 
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with  live  ammunition  at  a  fraction  of  tha  coat. 

In  collective  training,  two-aided  free  play  battlefield  engagement 
■isolator*  have  been  developed.  One  auch  simulation,  REAL TRAIN ,  require* 
that  each  aoldier  of  two  opposing  force*  wear  specially  treated  number* 
on  their  person.  Bach  soldier's  rifle  la  equipped  with  a  special  sighting 
lens.  Upon  sighting  an  opponent,  a  soldier  shouts  the  opponent's  number 
to  a  referee  who  records  a  "kill"  and  makes  sure  the  victim  retires  from 
the  game.  The  Multiple  Integrated  Laser  Bngagement  System  (MILES)  incorp¬ 
orates  the  LASER  simulator  in  a  collective  two-sided  free  play  exercise. 
Each  soldier  wears  a  belt  with  laser  sensors.  A  loud  tone  Indicates  to 
the  soldier  that  he  is  the  victim  of  a  hit.  If  hit,  the  aoldier  must  go 
to  a  referee  who  can  turn  off  the  tone  with  a  key.  A  softer  tone  Indicates 
to  the  soldier  that  he  was  the  subject  of  a  near  miss.  MILES  and  RBALTRAIH 
isolators  have  also  been  developed  for  tank  versus  tank  and  soldier  versus 
tank  simulators.  Further  developments  have  been  aimed  towards  integrating 
ARTEPs  with  REALTRAIN  and/or  MILES  to  obtain  tha  specificity  of  the  SQT 
evaluation  of  individuals  at  the  collective  training  luvel. 

This  survey  of  the  TRADOC  efforts  to  modernise  Army  training 
is  far  from  complete  but  is  indicative  of  the  actions  being  taken  to  meet 
this  goal. 

Constraints  on  TRADOC 

TRADOC  has  been  actively  pursuing  these  activities  despite 
shortages  of  personnel  and  resources.  Another  major  constraint  encoun* 
tered  Initially  by  TRADOC  was  the  varying  degree  of  resistance  to  change 
in  the  training  environment.  Thus,  TRADOC  is  in  the  position  of  not  only 
managing  a  very  large  and  complex  training  system,  but  also  is  actively 
attempting  to  modernise,  even  revolutionise.  Army  training  in  the  face  of 
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limited  resources  and  resistance  to  change.  Further,  because  there  ts  a 
strong  sense  of  time  urgency  in  accomplishing  these  tasks,  the  management 
of  TRADOC' s  planned  activities  is  even  more  arduous. 

It  is  within  this  context  that  TRADOC  has  contracted  SofTech,  Inc. 
to  apply  its  Structured  Analysis  and  Design  Technique  (SADT)  to  the  problems 
of  managing  rapidly  occurring  innovations  in  training  methods.  SADT  was  to 
be  used  in  this  study  to  identify  changes  in  Army  training  required  to 
significantly  increase  combat  effectiveness,  describe  how  Army  training, 
testing  and  evaluation  programs  would  operate  after  the  proposed  changes 
are  accomplished,  and  to  plan  how  the  changes  in  Army  training  would  be 
undertaken  and  how  progress  would  be  monitored  and  evaluated. 

Traditionally,  Army  training  has  not  been  viewed  as  a  large  system 
problem,  but  rather  as  a  composite  of  many  smaller  problems.  These  smaller 
problems  usually  have  been  solved  by  a  particular  organization  within  the 
training  command.  A  "good"  solution  often  optimized  the  particular  organ¬ 
ization's  objectives  at  some  expense  to  the  goal  >f  achieving  overall 
improvements  in  Army  training.  The  major  hypothesis  of  this  rtudy  was 
that  defining  military  training  as  a  large  system  will  provide  the  basis 
for  more  effectively  solving  small  problems  within  the  context  of  overall 
military  objectives. 

More  specifically,  TRADOC  participated  in  this  study  to  improve 
their  analysis  and  planning  of  the  training  system.  Special  emphasis  is 
given  by  TRADOC  to  the  problem  of  describing  the  interrelationships  between 
the  many  training  innovations  being  envisioned  and  in  planning  how  those 
changes  should  be  accomplished.  Once  developed,  it  is  hoped  that  the  model 
of  the  Army  training  system  can  be  used  to  guide  and  control  system 
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development  from  its  current  Jtatus  and  capability  to  the  status  and  capa¬ 
bility  desired.  After  the  training  system  is  fully  implemented,  the  model 
could  potentially  be  used  to  control  and  manage  the  system.  Finally,  the 
model  produced  could  become  the  "standard"  against  which  to  compare  current 
system  functioning.  Faced  with  the  problems  cited  earlier,  and  in  hopes 
of  achieving  the  goals  described  here,  TRADOC  entered  into  the  joint  pro¬ 
ject  with  SofTech  and  ARPA,  which  is  the  subject  of  this  evaluation  report. 
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IV.  Description  of  Project  Tasks  and  the  Evaluation  TIm  Prase 

Prior  to  actually  applying  SADT  to  Any  training  and  testing 
programs,  "lead  tine"  vas  allowed  for  establishing  a  polnt-of-contact  or 
liason  officer  within  TRADOC  to  function  as  an  interface  between  TRADOC 
and  SofTech.  This  person  was  to  be  responsible  for  setting  up  facilities, 
establishing  appointment  schedules,  and  coordinating  SofTech  and  TRADOC 
efforts.  A  aajor  portion  of  this  preparatory  activity  was  devoted  to 
training  selected  TRADOC  personnel  in  reading  and  authoring  SADT  diagraas. 
These  individuals  were  scheduled  to  play  an  active  role  in  most  project 
tasks,  particularly  as  coamenters. 

Project  Tasks 

The  plan  for  applying  SADT  in  the  TRADOC  environment  consisted 
of  five  project  tasks.  A  description  of  these  tasks  as  originally  proposed 
appears  below.  As  the  project  proceeded,  it  became  necessary  to  alter 
Tasks  4  and  5.  These  changes  will  be  discussed  in  a  subsequent  section 
of  this  report. 

Task  1.  In  this  task,  SofTech  analysts  were  to  produce  a 
model  describing,  at  the  overview  level,  how  the  Army  training  program 
would  function  after  the  training  Innovations  envisioned  by  TRADOC  were 
actually  incorporated  into  Army  practice.  The  principal  purpose  of  this 
initial  effort  was  to  define  the  types  of  changes  being  planned  by  TRADOC 
and  to  determine  how  these  changes  were  related  to  existing  training  methods. 
This  initial  description  was  to  provide  a  focus  for  Task  2  work  and  serve 
as  the  basis  for  Task  3. 

Task  2.  In  this  task  SofTech  analysts  and  TRADOC  officers 
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trained  In  SADT  were  to  develop  jointly  •  description  of  a  total  weapons 
systea.  The  objective  of  this  task  was  the  derivation  of  the  training 
requirements  necessary  for  that  weapon  system  to  be  maximally  effective 
on  the  battlefield.  It  was  believed  that  only  by  looking  at  a  particular 
weapons  systea  as  a  total  systea  could  that  systea' s  training  requirements 
be  fully  understood. 

The  Army '8  major  weapons  systems,  such  as  attack  helicopters, 
field  artillery  and  tanks,  are  necessarily  conceptualized  t  dependent 
components  within  a  larger  integrated  combined  arms  force.  Because  the 
tank  force  is  a  relatively  independent  system  requiring  an  increasing 
amount  of  technical  training  for  its  crew  and  support  elements,  it  was 
chosen  as  the  focus  of  Task  2.  The  results  of  a  thorough  analysis  of  the 
training  support  necessary  for  this  weapon  system  was  expected  to  be 
representative  of  what  would  be  found  in  analyses  of  other  systems.  The 
insights  gained  in  working  through  an  SADT  model  of  this  particular 
system  were  expected  to  be  of  value  in  determining  the  optimum  structure 
of  the  entire  Army  training  and  evaluation  program,  which  was  the 
focus  of  Task  3. 

Task  3.  In  Task  3  SADT  and  TRADOC  personnel  were  to  develop 
Jointly  an  SADT  model  of  the  new  Army  training  and  evaluation  systea  as  it 
should  function  after  training  innovations  envisioned  by  TRADOC  were  actually 
incorporated  into  the  Army's  standard  operating  procedures.  The  major 
input  to  this  effort  was  to  be  the  general  overview  model  of  the  Army 
training  systea  developed  in  Task  1,  and  the  SADT  model  and  associated 
reports  of  the  analysis  of  the  tank  system  as  a  total  weapon  systea  developed 
in  Task  2. 
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The  result  of  Task  3  was  to  be  a  Model  showing  in  increasing 
levels  of  detail  all  of  the  activities  that  comprise  the  training  system 
of  the  future.  These  activities  were  ho  include  such  tasks  as  course 
development,  budgeting  for  the  Arsj  schools.  Measuring  training  effective¬ 
ness,  determining  avenues  cf  information  flow  and  feedback  between  the 
various  organisational  coaponenta  of  the  training  system,  evaluating  the 
Impact  of  information  and  feedback  on  the  training  activities  that  actually 
occur,  measuring  cost  effectiveness,  and  evaluating  the  new  conceptualisation 
of  training  Itself. 

It  was  expected  that  this  aodel  would  facilitate  a  better  under¬ 
standing  of  what  the  Army  was  to  achieve  and  reduce  the  aaount  of  time 
spent  planning,  managing,  and  accomplishing  these  achievements.  Additionally, 
it  was  believed  that  the  aodel  would  provide  a  format  which  would  facilitate 
comnunlcatlon  to  and  from  the  various  proponent  schools  and  upper  echelons 
of  the  Department  of  the  Army. 

Task  4.  The  objective  of  this  task  was  to  be  the  development 
of  an  innovation  plan  describing  how  changes  in  Army  training  would  be 
undertaken  and  how  their  progreas  would  be  monitored  and  evaluated.  This 
plan  was  to  be  prepared  in  two  complementary  forms:  an  SADT  planning  model 
which  shows  precisely  the  interactions  between  the  various  elements  of  the 
plan  and  an  implementation  schedule  prepared  by  the  Crltial  Path  Method 
(CPM) . 

SofTech  was  to  assist  TRADOC  staff  members  in  developing  a 
model  of  the  Innovation  plan  and  was  to  produce  the  CPM  schedule.  The 
SofTech  deliverable  was  to  be  a  report  presenting  the  model,  the  resulting 
CPM  schedule,  and  a  summary  of  the  algorithms  used  to  perform  the  sodel 
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to  CPM  conversion 


Task  S.  This  task  was  to  consist  of  two  distinct  parts,  the  overall 
evaluation  of  this  project  (which  is  represented  by  this  report)  and  a  plan 
for  wider  Department  of  Defense  (DoD)  use  of  SADT  which  was  to  be  prepared 
by  SofTech  with  the  assistance  of  the  project  evaluator.  The  overall  eval¬ 
uation  was  to  focus  on  whether  changes  in  Any  training  actually  occurred 
as  a  result  of  the  project  and  whether  these  changes  could  increase  the  coabat 
readiness  and  battlefield  effectiveness  of  the  Amy.  This  evaluation  was 
to  include  an  SADT  model,  produced  by  the  evaluators,  of  the  overall  evalu¬ 
ation  effort. 

If  the  final  evaluation  indicated  that  the  project  goals  were 
net,  SofTech  was  to  prepare  a  plan  for  wider  DoD  use  of  SADT,  addressing 
four  key  issues: 

the  types  of  applications  where  the  DoD  could 
realize  the  greatest  benefits  and  an  estimate 
of  the  magnitude  of  the  benefits; 

the  types  of  documentation  and  courses  required 
if  a  significant  number  of  people  were  to  learn 
the  technique; 

areas  where  the  methodology  should  be  enhanced 
to  inprove  the  quality  of  the  results  or  its 
transferability  to  a  wider  audience; 

the  types  of  automated  tools  required  for  the 
exploitation  of  the  methodology  on  large  projects. 

SofTech  expected  that  the  evaluation  process  would  indicate  areas 
in  which  there  were  problems  in  applying  SADT  methodology  at  TRADOC,  areas 
where  the  technique  must  be  further  refined,  and  areas  where  alternative  methods 
might  be  preferable.  The  plan  was  to  indicate  how  any  deficiencies  in 
the  methodology  might  be  corrected  prior  to  wider  DoD  use. 


Evaluation  Time  Tr 


Tha  avaluation  of  thia  projact  was  divided  Into  three  phases, 
with  tha  raaulta  of  each  phase  praaantad  in  an  avaluation  report.  These 
phases  are: 


Phase  1.  This  phase  covered  Tasks  1  and  2,  and 
concentrated  on  determining  the  usefulness  of 
SADT  in  modeling  Army  training.  A  major  portion 
of  this  effort  was  the  identification  and  development 
of  evaluative  dimensions  on  which  tha  effectiveness 
of  SADT  and  the  usefulness  of  the  models  produced 
were  to  be  assessed.  The  evaluation  report,  submitted 
six  weeks  after  the  conclusion  of  Task  2  was  to 
serve  as  input  in  datermining  whether  or  not  to 
continue  the  project  as  planned. 

Phase  2.  This  evaluative  phase  was  concerned  pri¬ 
marily  with  determining  the  impact  of  the  Task  3 
model.  Because  the  model  produced  in  Task  3  was  to 
represent  the  bulk  of  SofTech's  work  for  TRADOC, 
an  evaluation  of  tha  perceived  usefulness  and  Impact 
of  thia  particular  model  seemed  appropriate  prior 
to  beginning  the  final  (thase  3)  evaluation  effort. 
Further,  an  Interim  evaluation  deliverable  at  the 
conclusion  of  Task  3  would  provide  an  opportunity 
to  reconsider  and  update  the  findings  and  conclu¬ 
sions  of  the  Phase  1  evaluation. 

Phase  3.  This  evaluative  phase  was  to:  (1)  re¬ 
examine  the  results  and  conclusions  of  the  first 
two  evaluative  deliverables,  and  (2)  focus  upon 
the  intermediate  and  long  range  effects  of  having 
developed  the  models  produced  in  Tasks  1,  2,  3, 
and  4.  Primary  emphasis  was  given  in  this  phase 
to  whether  the  models  had  brought  about  a  better 
understanding  of  what  needed  to  be  done  to  Improve 
Army  training,  whether  TRADOC  personnel  considered 
SADT  a  viable  approach  to  their  problems  and  * 
determination  of  whethar  tha  work  produced  during 
this  project  would  ultlmetely  make  a  difference  in 
Army  training  and  evaluation  procedures.  This 
paper  constitutes  the  final  evaluation  report  of 
this  project  and  susmarises  the  procedures  and  results 
of  all  three  evaluation  phases. 

Figure  4  indicates  the  relationship  between  project  tasks 


and  evaluation  phases. 


Project  Slzari: 
(April,  1976) 


tasks 


results  and  conclusions  for  each  of 


V.  Phase  1  Evaluation  Procaduraa  and  Results 


General  Approach. 

The  propoaal  aubaitted  to  ARPA  by  SofTech  apeclfled  an  evaluation 
effort  which  would  provide  ARPA  with: 

(1)  an  aaaeaaMnt  of  the  utility  and  effectiveness 
of  the  Structured  Analysis  and  Design  Technique 
(SADT)  in  the  training  cosmand  environment;  and 

(2)  an  evaluation  of  the  impact  of  the  project  on 
TRADOC' a  ability  to  identify  the  types  of  changes 
required  in  Army  training  and  to  develop  a  practical 
plan  for  carrying  out  those  changes. 

The  general  goals  of  the  evaluation  effort  (1  and  2  above)  and 
the  experience  accrued  by  the  evaluators  during  an  orientation  visit  to 
TRADOC  Headquarters  suggested  two  major  areas  of  inquiry  for  the  Phase  1 
evaluation.  The  first  addressed  the  utility  and  effectiveness  of  the 
technique  itself  and  encompassed  such  questions  as:  Are  SADT  diagrams 
accurate?  Does  the  SADT  process  promote  comunication  between  diverse 
personnel  in  che  project?  Does  SADT  help  achieve  clearer  understanding 
of  the  problems  at  hand? 

The  second  major  area  of  inquiry  focused  on  the  Interface  between 
the  technique  and  the  particular  environment  in  which  it  is  applied,  in 
this  case,  TRADOC.  The  value  of  the  technique  itself  is  inconsequential  if 
the  technique  is  not  successfully  implemented.  Because  SADT  is  dependent 
on  the  cooperation  and  support  of  personnel  in  the  client  organisation, 
marginal  implementation  in  an  unreceptive  environment  or  a  lack  of  uniform 
support  could  affect  the  utility  and  effectiveness  of  the  technique  itself. 

Thus,  two  evaluation  questions  emerged.  The  first  addressed  the 
value  of  SADT;  the  second  addressed  the  value  of  the  technique  as  applied 
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in  an  environment  like  TRADOC.  This  second  question  was  dependent  on 
the  degree  of  implementation  and  the  amount  of  support  the  technique 
received  from  the  client  organization,  TRADOC.  Degree  of  implementation 
could  be  measured  by  various  behavioral  Indices  of  participation  in  the 
project  by  members  of  TRADOC.  The  value  question  was  not  so  easily 
answered.  Given  the  available  resources  and  time  frame  for  the  present 
evaluation  effort,  the  perceived  effectiveness  of  the  technique  by  partici- 
pating  TRADOC  officers  was  given  primary  consideration  in  determining  the 
value  of  SADT. 

The  remainder  of  this  section  attempts  to  answer  these  questions 
by  specifying  the  evaluation  methods  and  procedures  followed,  reporting 
and  discussing  the  results  obtained,  presenting  conclusions  based  on  the 
results,  and  providing  a  summary  of  critical  issues  which  affected  the 
implementation  of  SADT  in  the  TRADOC  environment. 

Methods  and  Procedures 

A.  Site  visits.  The  evaluation  team  (Dr.  Gary  D.  Borich  and 

Mr.  Ron  Jemelka)  visited  SofTech's  Waltham,  Massachusetts  facility  and  TRADOC 
Headquarters  at  Ft.  Monroe,  Virginia  in  August,  1976.  These  visits  provided 
inital  contact  with  SofTech  and  TRADOC,  and  provided  a  realistic  framework 
from  which  to  plan  subsequent  evaluation  efforts.  Additionally,  the  eval¬ 
uation  staff  was  provided  training  in  reading  and  authoring  SADT  diagrams 
during  these  site  visits. 

B.  The  development  of  evaluative  dimensions.  The  next  major 
evaluation  effort  was  the  development  of  evaluative  dimensions  to  provide  a 
framework  for  the  assessment  of  SADT.  The  rationale  for  developing  the  evaluative 
dimensions  was  taken  from  the  technical  proposal  submitted  to  ARPA  by  SofTech. 
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The  approach  used  in  developing  these  dimensions  was  to  analyte  the 
benefits  claimed  by  SofTech  to  occur  at  TRADOC  as  a  result  of  this 
contract.  Because  the  selection  of  the  evaluative  dimensions  was  critical 
to  subsequent  evaluation  efforts,  these  dimensions  were  derived  directly 
from  the  contractor's  claims,  to  protect  the  evaluation  from  being  biased 
by  criteria  which  were  either  unfair  to  the  contractor  or  insensitive 
to  Army  needs  and  expectations.  Deriving  dimensions  from  claims  made  by 
the  contractor  in  the  technical  proposal  seemed  ultimately  fair  to  both 
SofTech  and  TRADOC,  because  SofTech  wrote  the  proposal  (made  the  claims) 
and  TRADOC  accepted  the  proposal  as  documentation  of  what  was  to  be 
delivered.  The  only  constraint  imposed  by  the  evaluator  in  selecting 
evaluative  criteria  was  that  each  dimension  chosen  had  to  be  measurable, 
given  the  resources  and  time  frame  allotted  to  the  evaluation  effort. 

The  technical  proposal  was  analyzed  and  each  claim  was  noted.  The 
process  of  deriving  evaluative  dimensions  began  by  clustering  similar 
claims  together.  After  this  first  clustering,  each  cluster  was  scrutinised 
further  to  determine  if  (a)  there  were  significant  overlap  between  clusters, 
(b)  there  were  only  a  few  claims  in  any  one  cluster,  and  (c)  there  were 
radically  different  claims  in  any  one  cluster.  In  some  cases,  clusters  were 
combined  and,  in  ether  cases,  clusters  were  broken  into  related  subclusters. 
The  overriding  criterion  for  the  composition  of  clusters  was  that  each  be 
as  independent  of  the  others  as  possible. 

Each  cluster  was  then  analyzed  and  a  generic  title  chosen  to  cover 
all  claims  in  that  cluster.  In  this  manner,  each  cluster  evolved  into  an 
evaluative  dimension.  These  evaluative  dimensions  appear  in  Appendix  A. 
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C.  Data  gathering  strategies.  Given  the  time  and  resource  limita¬ 
tions  of  the  evaluation  effort,  the  most  direct  manner  in  which  to  evaluate 
SADT  in  Tasks  1  and  2  was  via  the  perceived  effectiveness  of  the  technique 
by  TRADOC  personnel  involved  in  the  project.  The  most  straightforward 
strategy  would  be  to  ask  these  individuals  about  various  aspects  of  the 
technique  relevant  to  the  evaluative  dimensions  and  what  benefits  they 
believe  have  accrued  or  could  accrue  to  the  Army  as  a  result  of  its  partici¬ 
pation  in  the  project.  Therefore,  a  questionnaire  was  developed  and  admin¬ 
istered  to  TRADOC  personnel  and  then  followed  with  a  structured  interview 
focusing  on  specific  responses  to  the  questionnaire.  The  interview  also 
provided  the  opportunity  to  pose  general  questions  about  the  value  of 
the  technique  and  the  use  of  the  technique  in  the  TRADOC  environment.  A 
copy  of  the  questionnaire  appears  in  Appendix  B. 

According  to  SofTech's  descriptive  literature,  one  of  the  necessary 
"subsystems"  of  an  SADT  project  is  a  file  system  and  library  service  which 
organizes  and  centralizes  record-keeping  and  support  functions  and  which 
allows  the  complete  project  history  to  be  reviewed.  This  clerical  function 
ensures  that  documentation  of  the  development  of  an  SADT  model  is  maintained 
concurrent  with  that  development.  A  copy  of  each  diagram  produced  it. 
retained  on  file,  and  all  interactions  between  individuals  in  the  project 
are  documented. 

This  project  file  makes  it  possible  to  determine  the  subject  area 

of  a  kit  (a  series  of  related  SADT  diagrams) ,  number  of  diagrams  in 
the  kit,  who  the  kit  was  sent  to  for  comments,  how  many  times  kits  were 
commented  on  and  returned,  and  how  often  the  author  returned  the  kit  to 
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the  reader.  From  these  data,  one  can  determine  whether  the  activities 
described  in  the  proposal  actually  occurred.  While  designed  to  enhance 
the  review  of  project  decisions  and  development,  these  data  were  important 
in  determining  if  the  technique  had  been  successfully  implemented  at 
TRADOC  headquarters. 

The  project  file  at  Ft.  Monroe  was  examined  to  determine  if  this 
library  function  had  been  maintained  as  claimed.  The  number  of  diagrams 
produced  and  general  content  of  the  diagrams  were  noted.  These  data 
then  were  used  to  prepare  frequency  tables  for  all  relevant  process  behaviors. 
TRADOC  participants  were  queried  at  random  to  determine  the  degree  of 
agreement  between  perceived  level  of  participation  and  degree  of  participation 
documented  in  the  project  file. 

D.  Summary  of  procedures.  The  Phase  1  evaluation  procedures 
yielded  three  types  of  data: 

(1)  process  data  -  obtained  from  examination  of  the  project  file; 

(2)  quantitative  data  -  obtained  from  the  quantifiable  question¬ 
naire  responses  (see  Appendix  B,  Items  1  through  38); 

(3)  qualitative  data  -  obtained  from  responses  to  open-ended 
questionnaire  items  (Items  39  through  44)  and  from  responses 
to  Interview  probes. 

After  the  results  of  each  type  of  data  were  prepared  they  were 
combined  where  appropriate  to  answer  the  two  basic  questions  which  were 
the  focus  of  the  Phase  1  evaluation  effort.  The  first  question  concerned 
the  value  or  merit  of  SADT  itself.  The  second  question  addressed  the 
implementation  of  SADT,  particularly  in  an  environment  such  as  TRADOC. 
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The  data  from  all  three  aourcea  were  interrelated  i  >-ior  to  drawing  conclu¬ 
sions  about  the  value  of  SADT.  The  interaction  of  hese  three  groupa 
of  data  illuminated  both  the  technique  itaelf  and  the  effect  of  variablea 
which  moderate  the  ef fsctiveness  of  the  technique. 

Re8ults 


Thi8  section  presents  results  of  the  Phase  1  evaluation.  Data 
on  the  iapleaentation  of  SADT  during  this  phase  will  be  presented  first 
followed  by  results  from  the  questionnaire  and  structured  interviews. 

A.  Process  data.  The  first  step  in  this  phase  of  the  evaluation 
was  to  determine  whether  the  roles  of  the  TRADOC  and  SofTech  personnel 
specified  in  the  contract  proposal  had  actually  been  performed.  Examination 
of  what  had  occurred  up  to  this  point  in  the  project  revealed  that  the 
processes  and  role  functions  specified  in  the  proposal  did  occur  with 
some  exceptions.  One  notable  ommission,  however,  was  that  no  Senior  Review 
Committee  was  reported  in  the  Task  1  report,  SofTech  explained  the  absence 
of  the  Senior  Review  Committee  by  noting: 

As  fewer  changes  were  indicated  by  consents 
received,  the  diagrams  would  normally  have  been 
sent  to  a  selected  group  of  commenters  referred 
to  as  the  Senior  Review  Committee.  This  committee 
would  be  requested  to  review  these  diagrams  and 
approve  them.  Since  the  commenters  included  the 
members  of  the  review  comittee,  and  since  this 
model  will  be  revised  in  Task  3,  the  formal  senior 
review  was  omitted.  (SofTech,  Inc.,  Task  1  Report,  p.  3-5) 

A  second  discrepancy  between  contractor  intents  and  conditions 
observed  by  the  evaluators  was  that  some  individuals  performed  multiple 
roles.  For  example,  the  technical  proposal  specified  that  the  individual 
assigned  as  the  TRADOC  interface  was  to  be  responsible  for  time  commitments. 
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selection  of  staff  for  training  and  participation,  and  for  ensuring  comple¬ 
tion  of  TRADOC  deliverables.  Although  these  functions  were  performed 
initially  the  role  was  modified  early  on  in  the  project  and  ultimately 
became  nonexistent.  This  individual  was  given  other  assignments  unrelated 
to  his  responsibilities  as  the  TRADOC  interface  and  was  asked  to  perform 
other  duties  within  the  project  that  limited  the  amount  of  time  that  could 
be  devoted  to  the  interface  task.* 

Another  point  concerns  the  sources  of  information  for  SADT 
authors.  Although  the  number  of  experts  interviewed  was  at  least  ten  (as 
specified  in  the  proposal),  discussion  with  SADT  authors  indicated  that 
early  in  the  modeling  process,  significant  information  was  gained  by 
reading  technical  and  progress  reports  on  training  and  the  proceedings 
of  several  conferences  on  training  sponsored  by  TRADOC.  Thus,  the  SADT 
process  was  not  Implemented  initially;  but  rather  following  a  schedule 

of  background  reading  which  undoubtedly  made  the  subsequent  author-TRADOC 
interviews  more  meaningful.  This  seems  to  be  a  desirable  and  natural 

activity  prior  to  modeling  but  no  mention  is  made  of  it  in  the  SADT  authoring 
procedures. 

Examination  of  the  events  surrounding  Task  2  of  this  project 
revealed  several  discrepancies  from  those  specified  in  the  technical 
proposal.  The  most  notable  involves  the  full  time  assignment  to  Task  2 
of  two  TRADOC  personnel  as  SADT  authors.  The  SADT  training  provided  these 
individuals  was  not  timely,  or  necessarily  complete  relative  to  the  Task  2 


♦Another  Individual  with  several  functions  waB  the  on-site  project 
manager  for  SofTech.  In  addition  to  management  responsibilities,  he 
was  one  of  the  two  authors  provided  by  SofTech  for  Task  1.  He  also 
assumed  some  of  the  functions  of  the  TRADOC  Interface  when  necessary. 


•ffort.  Only  one  TRADOC  author  received  formal  author  training,  thin  oc¬ 
curring  during  the  second  half  of  Task  2.  Although  the  second  individual 
received  some  on-the-job  training  to  prepare  him  to  author  diagrams,  this 
training  was  reported  by  the  two  SofTech  authors  on  site  to  be  minimal . 

Other  than  those  noted  above,  the  prescribed  roles  listed  for 
Tasks  1  and  2  in  the  technical  proposal  were  carried  out  as  specified. 

Examination  of  SofTech' s  descriptive  literature  reveals  that 
several  related  classes  of  behavior  constitute  the  SAD?  process  or  disci¬ 
pline.  Two  of  these  are  the  iterative  author-coassenter  cycle,  which  is 
the  vehicle  for  communication  between  SADT  authors  and  conmenters  in  the 
client  organization,  and  maintenance  of  the  SADT  project  library,  which 
provides  complete  docuawntatlon  of  the  project  history. 

The  claims  made  by  SofTech  imply  benefits  such  as  enhanced  communi¬ 
cation,  understanding,  and  involvement  by  personnel  in  the  client  organi¬ 
zation,  and  casts  SADT  as  a  heuristic  device  which  will  aid  in  the  solution 
of  problems.  The  most  central  element  of  the  SADT  process  is  comaenting 
on  a  kit.  Because  commenting  on  diagrams  is  the  only  official  contact 
between  SofTech  personnel  and  individuals  in  the  client  organisation, 
this  activity  must  logically  occur  before  any  of  the  above-mentioned 
benefits  accrue. 

Table  2  presents  data  on  kit  issuance  and  return  by  rank  of 
commenter  for  Tasks  1  and  2.  All  the  data  presented  were  obtained  from 
the  Task  1  and  Task  2  reader  kit  coveroheets  in  the  project  library  file. 
Column  11  represents  the  number  of  kits  each  reader  was  expected  to 
comment  on.  Column  111  indicates  the  number  of  times  the  reader  actually 
contented  on  and  returned  the  kit  to  the  author  by  the  time  prescribed. 
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Coluan  IV  rtprucnta  those  instances  In  which  s  kit  was  returned  but  lata. 
Coluan  V  indicates  the  nuaber  of  tlaes  a  kit  was  sent  to  n  render  for 
coarsen ts  but  never  returned  to  the  author.  Coluan  VI  indicates  the  nuaber 
of  tlaes  it  was  unclear  as  to  what  actually  transpired  with  any  one  kit , 
i.e.,  it  was  not  possible  to  discern  whether  the  kit  had  been  consented 
on  or  not.  This  is  clearly  a  failure  of  the  librarian  function  to  docuaent 
precisely  what  had  transpired. 

Totals  at  the  bottoa  of  Table  2  indicate  that  104  or  54Z  of  the 
kits  issued  to  readers  for  coa*ents  were  not  returned  to  the  author. 

This  level  of  participation  by  TRADOC  personnel  in  the  Iterative  author- 
coionenter  cycle  of  SADT  was  considerably  less  than  SofTech  had  originally 
anticipated. 

The  large  nuaber  of  kits  not  returned  led  the  evaluators  to  ask 
a  SofTech  project  aeaber  about  these  discrepancies.  It  was  Indicated 
that  In  the  later  stages  of  the  node ling,  the  interaction  between  SADT  autnors 
and  the  Senior  Review  Coonlttee  was  often  verbal  and  final  approval  of 
diagrams  was  obtained  in  a  conference  setting.  According  to  SofTech 
descriptions,  however,  a  conference  between  coanenter  and  author  is 
reserved  for  the  case  in  which  differences  cannot  be  resolved  on  paper 
and  all  Interactions,  for  whatever  purpose,  are  to  be  recorded.  The 
procedure  used  appears  to  be  at  variance  with  SofTech' s  descriptive 
literature. 

In  atteaptlng  to  explain  this  discrepancy  fro*  standard  procedure, 
a  SofTech  project  *e*ber  explained  that  working  in  this  snvironaent  was 
at  tines  too  hectic  for  hi*  to  perfior*  clerical  duties.  Another  SADT 
author  stated  that  to  his  knowledge,  he  logged  all  required  data  on  kits 
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nters  by  Selected  Process  Behaviors  for  Tasks  1  and  2 


This  coluan  represents  the  nuaber  of  tiaes  it  was  unclear  froa  the  coversheet  Information  whether 
the  kit  was  returned  to  the  author  by  the  connenter  and  appears  to  be  the  result  of  poor  book¬ 
keeping.  The  total  of  coluans  3,  4,  5  and  6  should  equal  coluan  2. 


he  authored  and  Issued,  but  added  that  performing  these  functions  detracted 
froa  his  authoring  activities.  When  asked  specifically  about  two  indi¬ 
viduals  for  whoa  records  indicated  they  had  returned  less  than  10X  of  tha 
kits  issued  to  than,  the  evaluators  received  verification  that  the  figure 
seemed  valid.  Despite  the  tendency  of  SofTech  authors  not  to  record 
final  stage  approvals  of  their  diagrams,  it  seems  reasonable  to  accept 
the  reader  kit  cover  sheets  as  estimates  of  the  interactions  that  went  on 
during  the  project. 

Another  area  where  SofTech 'a  original  expectations  of  TRADOC 
participation  was  not  met  was  in  the  assignment  of  two  TRADOC  personnel 
who  were  to  becosw  authors  for  Task  2.  This  assignment  seemed  critical 
to  TRADOC' s  "internalising"  the  SADT  procedures.  The  fact  that  the  indi¬ 
viduals  were  not  provided  in  a  way  that  they  could  both  be  fully  trained 
and  assigned  to  the  SofTech  project  full-time  as  expected  raises  two  issues. 

The  first  pertains  to  the  productivity  of  project  personnel 
during  Task  2.  Given  that  one  of  the  individuals  assigned  was  never 
formally  trained  and  the  other  was  trained  three  months  after  the  Task  2 
effort  was  begun,  it  would  be  unreasonable  to  expect  the  same  results 
and  degree  of  Internalization  of  the  technique  had  initial  expectations 
been  met.  To  quote  the  program  manager  of  this  project,  "SofTech  certainly 
would  have  been  much  more  guarded  in  predicting  the  impact  of  the  work 
’’’RADOC  if  the  actual  level  of  Army  author  participation  could  have 
been  predicted  at  the  time  the  proposal  was  written." 

The  second  issue  arising  from  TRADOC' s  failure  to  provide  the 
m;  /er  originally  agreed  upon  concerns  the  evaluation  of  the  transfer 
c.  "Ms  technology  from  SofTech  to  TRADOC.  Determining  the  extent  to 
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which  Army  personnel  can  be  trained  In  SADT  methodology  seems  critical 
to  determining  the  applicability  of  tha  technique  to  other  environments 
within  the  Department  of  Defense.  Unfortunately,  only  one  author  was 
trained.  Hhile  this  individual  was  considered  a  good  SADT  author  by  SofTech 
nemnnel,  he  was  viewed  by  his  superiors  at  TRADOC  as  a  unique  individual 
with  considerable  talent  and  potential,  especially  in  the  area  of 
operations  research.  One  officer  stated  during  an  interview  that  this 
individual  "is  not  representative  of  the  population  at  TRADOC."  Certainly 
there  are  competent  individuals  who  can  learn  the  technique,  but  their 
productivity  in  using  SADT  to  model  program  functions  and  the  range  of 
individuals  capable  of  learning  the  technique  In  DoD  environments  cannot 
be  determined  from  the  present  data. 

In  summary,  the  process  data  collected  indicates  that  the  SADT 
process  was  not  always  implemented  as  specified  at  TRADOC  through  Task  2. 
Participation  by  TRADOC  personnel  in  the  iterative  author-commenter  cycle 
was  approximately  half  that  expected.  This  reduced  availability  of  TRADOC 
personnel  for  training  and  authoring  of  SADT  diagrams  limited  the  potential 
to  institutionalize  the  technique  at  TRADOC.  Thus,  final  and  definitive 
conclusions  about  the  success  of  transferlng  this  technology  to  a  DoD 
environment  cannot  be  made  from  these  data. 

B.  Quantitative  data.  This  section  reports  responses  to 
multiple  choice  and  Likert  type  questionnaire  items  given  to  TRADOC 
officers  working  with  the  project.  The  questionnaire  was  completed 
by  13  of  16  Task  1  and  2  cosmenters.  Statistical  calculations  indi¬ 
cate  that  the  questionnaire  reliably  measured  some  characteristic  of 
the  respondents  and  that  the  individual  items  were  producing  similar 
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patterns  of  responding  in  different  Individuals  (  a  ■  .93).*  The  a> 


was  true  for  evaluative  diaenaiona  11^  (a  -  .86),  III^  (  a  »  .96)  and 
I1IB  (  a  -  .96)  and  to  a  leaser  extent  for  dimension  IIC  (  a  ■  . 71) . 

The  pattern  of  responses  for  diaenaiona  I.,  I_,  and  II_  were  aore  hater- 

A  a  o 

ogeneous  and  of  questionable  rellabllty  (a  •  .35,  .45,  and  .51  respect¬ 
ively)  . 

Table  3  presents  the  nuaber  of  respondents,  the  aean  (X)  and 
the  standard  deviation  (SD)  of  each  quantitative  ltea  on  the  questionnaire 
The  aajor  Intent  of  each  ltea  is  included  and  lteas  are  clustered  by  eval¬ 
uative  diaenslons  for  easy  reference. 

TWo  conclusions  can  be  made  froa  examination  of  questionnaire  data. 
The  first  concerns  the  value  of  SADT  as  an  "engineering  drawing  systea 
for  systeas  description."  This  view  of  SADT,  presented  by  SofTech  in  the 
technical  proposal,  stresses  standardised  graphics,  controlled  document 
revision  through  the  provision  of  procedural  mechanisms  for  audit  purposes 
and  effective  coaaunication  between  the  originator  and  user  of  descriptive 
models. 

The  results  of  Subscales  1^,  11^,  and  Ilg,  Indicate  that  as  an 
engineering  drawing  systea,  SADT  is  viewed  positively  by  TRADOC  personnel. 
The  general  consensus  is  that  the  SADT  discipline,  which  specifies  the 
format  of  all  coaaunicatlons,  controls  the  routing  of  diagrams  and 
establishes  the  time  fraae  in  which  these  behaviors  occur,  generally 
results  in  accurate,  highly  readable  diagraas  that  communicate  the 
substantive  components  of  coaplex  probleas. 

*  Alpha  (  a  )  reliabilities  calculated  with  the  standard  Kuder- 
Richardson  formula  (see  Guilford,  J.P.  Fundamental  statistics  in  psy¬ 
chology  and  education.  Hew  York:  McGraw-Hill,  1965,  458-460.) 
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Table  3 

Item  Means  and  Standard  Deviations  for  TRADOC  Headquarters  Coammntera 


Items 


N 


Standard 
Mean  Deviation 


I.  COMMUNICATIVE  QUALITY 


A.  of  Diagrams 

1.  Respondent  adequately  trained  to 


read  diagrams? 

13 

4.08 

0.95 

2. 

Respondent  adequately  trained  to 
content  on  diagrams? 

13 

3.92 

0.86 

3a. 

Ftvst  drafts  of  diagrams  clear  and 
unambiguous? 

12 

3.75 

0.87 

3b. 

Revised  versions  of  diagrams  clear 
and  unambiguous? 

13 

4.31 

0.48 

3c. 

Final  versions  of  diagrams  clear 
and  unambiguous? 

12 

4.58 

0.51 

4. 

Would  written  text  explaining 
diagrams  be  helpful?** 

12 

3.00 

1.21 

5. 

Should  discussion  be  a  part  of  the 

SADT  process?*** 

13 

4.38 

0.75 

6. 

Are  diagrams  an  effective  way  of 
communicating?* 

13 

4.23 

0.60 

10. 

Project  benefits  attributable  to 
diagrams  or  to  processes  stimulated?* 

9 

3.44 

0.53 

Mean  and  standard  deviation  for 

Subscale  1^ 

3.97 

0.75 

B. 

of  Process 

7. 

Have  diagrams  generated  communi¬ 
cation  among  individuals  involved 
(i.e„,  asked  to  be  commenters)  in 
project? 

13 

3.38 

1.45 

8. 

Have  diagrams  generated  communi¬ 
cation  among  individuals  not 

Involved  (i.e.,  not  asked  to  be 
commenters)  in  project? 

13 

1.85 

0.99 

9. 

Have  diagram  helped  focus  atten¬ 
tion  on  variables  not  considered 
previously?* 

13 

3.38 

1.26 

10. 

Project  benefits  attributable  to 
diagram  or  to  proceases 
stimulated?* 

9 

3.44 

0.53 
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Table  3 (continued) 


Standard 


Items 

N 

Mean 

Deviation 

30. 

Were  others  consulted  before  coa- 
nentlng  on  a  diagram?  ft 

12 

1.00 

0.00 

Mean  and  standard  deviation  for 

Subscale  1^ 

3.01 

0.85 

II.  QUALITY 

A. 

Accuracy 

6. 

Are  diagrams  an  effective  way  of 
connnunicat  ing?  * 

13 

4.23 

0.60 

11. 

Have  diagrams  accurately  represented 
content  modeled  in  Task  1? 

11 

4.09 

0.54 

12. 

Have  diagrams  accurately  represented 
content  modeled  in  Task  2? 

9 

4.00 

0.87 

14. 

Confident  that  author-connenter 
cycle  guarantees  accuracy  and 
completeness?* 

13 

4.08 

0.76 

Mean  and  standard  deviation  for 

Subscale  IIA 

4.10 

0.67 

B. 

Quality  Control 

13. 

Does  aulhor-commenter  cycle  ensure 
quality  of  diagrams? 

13 

4.31 

0.75 

14. 

Confident  that  author-commenter 
cycle  guarantees  accuracy  end 
completeness?* 

13 

4.08 

0.76 

15. 

Confident  that  conments  were  taken 
into  account? 

13 

4.77 

0.44 

28. 

Confident  in  comments  made  on 
diagrams? 

13 

4.00 

0.58 

29. 

Respondent  lacked  sufficient  expe¬ 
rience  to  make  meaningful  comments?** 

12 

3.42 

1.38 

Mean  and  standard  deviation  for 

Subscale  11^ 

4.12 

0.85 

C. 

Efficiency 

16. 

Has  communicating  via  author- 
conmenter  cycle  saved  time? 

13 

3.69 

1.03 

17. 

Has  SADT  saved  time  relative  to 
other  available  approaches  to  the 
problem? 

11 

3.36 

1.50 
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Table  3  (continued) 


I  teas 

N 

Mean 

Standard 

Deviation 

18. 

Is  SADT  cost-efficient  relative  to 
other  techniques  hsving  ssm  purpose? 

6 

3.67 

1.21 

19. 

Percentage  of  tine  saved  using  SADT 
on  design  and  analysis  task?t 

7 

0.26 

0.24 

37. 

Has  the  progress  made  been  worth  the 
tine  spent?* 

12 

3.50 

1.38 

Mean  and  standard  deviation  for 

Subscale  11^ 

3.56 

1.28 

III.  USEFULNESS 

A. 

In  Tasks  1  and  2 

9. 

Have  diagrams  helped  focus  attention 
on  variables  not  considered 
previously?* 

13 

3.38 

1.26 

20. 

Did  Task  1  model  provide  clearer 
understanding  of  Army  training? 

11 

3.09 

0.83 

21. 

Will  models  produced  thus  far 
actually  be  used  in  planning  changes? 

12 

2.92 

1.00 

22. 

Will  models  produced  thus  far 
actually  help  to  derive  training 
requirements? 

12 

3.17 

1.11 

24a. 

Will  SADT  increase  TRADOC's  analysis 
capability?* 

10 

3.60 

1.51 

24b. 

Will  SADT  Increase  TRADOC's  planning 
capability?* 

10 

3.40 

1.26 

24c. 

Will  SADT  increase  TRADOC's  manage¬ 
ment  capability?* 

10 

3.00 

1.05 

25. 

Can  SADT  help  identify  training 
requirements  affecting  combat 
effectiveness?* 

13 

3.77 

1.09 

27. 

Is  it  practical  for  TRADOC  to  use 

SADT  for  design  and  analysis 
problems?* 

13 

3.62 

1.12 

32. 

Will  SADT  models  provide  impetus  for 
changes  in  Army  training?* 

12 

3.17 

0.94 

33. 

Will  SADT  be  useful  in  identifying 
existing  organisational 
inefficiencies?* 

46 

12 

3.75 

0.62 
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Tabic  3  (continued) 


Items 


Standard 

N  Mean  Deviation 


34. 

Will  SADT  lead  to  conceptual  insights 
about  conducting  Anay  training?* 

13 

3.46 

0.88 

36. 

Would  it  be  useful  to  train  some  Army 
personnel  to  be  SADT  authors?* 

13 

3.69 

1.49 

37. 

Has  the  progress  made  been  worth  the 
time  spent?* 

12 

3.50 

1.38 

38. 

Did  Task  2  model  provide  clearer 
understanding  of  Army  training? 

11 

3.64 

1.03 

Mean  and  standard  deviation  for 

Subscale  III. 

A 

3.41 

1.10 

B. 

In  Subsequent  Tasks 

23. 

Will  SADT  be  effectively  used  to 
elaborate  Task  1  model  in  Task  3? 

13 

4.31 

0.63 

24a. 

Will  SADT  increase  TRADOC's  analysis 
capability?* 

10 

3.60 

1.51 

24b. 

Will  SADT  increase  TRADOC’s  planning 
capability?* 

10 

3.40 

1.26 

24c. 

Will  SADT  increase  TRADOC’s  manage¬ 
ment  capability?* 

10 

3.00 

1.05 

25. 

Can  SADT  help  identify  training 
requirements  affecting  combat 
effectiveness?* 

13 

3.77 

1.09 

27. 

Is  it  practical  for  TRADOC  to  use 

SADT  for  design  and  analysis 
problems?* 

13 

3.62 

1.12 

32. 

Will  SADT  models  provide  impetus  for 
changes  in  Army  training?* 

12 

3.17 

0.94 

33. 

Will  SADT  be  useful  in  identifying 
existing  organizational 
inefficiencies?* 

12 

3.75 

0.62 

34. 

Will  SADT  lead  to  conceptual  insights 
about  conducting  Army  training?* 

13 

3.46 

0.88 

35. 

Will  use  of  SADT  ultimately  affect 
training  concepts  held  by  TRADOC? 

13 

3.15 

0.80 

36. 

Would  it  be  useful  to  train  some  Array 
personnel  to  be  SADT  authors?* 

13 

3.69 

1.49 

Mean  and  standard  deviation  for 

Subscale  III^ 

3.54 

1.04 
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Table  3  (continued) 


Items 

N 

Mean 

Standard 

Deviation 

SPECIAL  SCALES 

Value  of  SADT  Model  as  a  Reference 

26.  Would  an  SADT  model  of  Army  training 
be  of  value  as  a  reference  document? 

*3 

3.85 

1.14 

Mean  and  standard  deviation  for  this 
subscale 

3.85 

1.14 

Information  Grasp 

28.  Confident  in  comments  made  on 
diagrams?* 

13 

4.00 

0.58 

29.  Respondent  lacked  sufficient  expe¬ 
rience  to  make  meaningful  comments?** 

12 

3.42 

1.38 

30.  Were  others  consulted  before  com¬ 
menting  on  a  diagram?  +f 

12 

1.00 

0.00 

Mean  and  standard  deviation  for  this 
subscale 

3.71 

0.62 

Desire  for  Change 

31.  Presently,  how  critical  is  it  to 

implement  changes  in  Army  training? 

13 

4.62 

0.65 

Mean  and  standard  deviation  for  this 
subscale 

4.62 

0.65 

Questionnaire  Total 

3.58 

0.93 

Item  also  represented  by  another  evaluative  dimension. 

Scoring  -reversed  for  this  item. 

This  item  had  only  four  alternatives,  scored  in  the  following  manner:  1*4, 
2  ■  5,  3  *  2,  4  ■  1. 

^This  item  was  not  in  multiple  choice  format.  The  mean  and  S.D.  for  this 
item  were  not  included  in  the  calculation  of  the  per  item  mean  and  S.D.  for 
Subscale  lig. 

++This  item  covers  a  communication  aspect  discouraged  by  the  methodology.  The 
mean  and  S.D.  for  this  item  were  not  included  in  the  calculation  of  the  per 
item  mean  and  S.D.  for  this  subscale. 


Scores  were  lower  for  the  evaluative  dissensions  that  inquired 
into  SADT's  impact  or  usefulness  at  TRADOC  (I  ,  III  ,  III.).  When 

O  A  0 

queried  about  SADT  diagrams  and  the  SADT  process  per  ae,  responses 
were  quite  positive  (The  item  means  for  evaluative  dimensions  1^,  11^, 
and  II  were  3.97,  4.10.  and  4.12  respectively  on  a  1  to  5  point  scale). 

D 

When  questions  addressed  the  application  of  SADT  within  the  TRADOC  context, 
resulting  scores  were  noticeably  lower  (2.61,  3.41,  and  3.54  for  evalu¬ 
ative  dimensions  IQ,  III^,  and  IIIg,  respectively).  This  difference 
between  the  context-free  and  context  specific  views  of  SADT  suggests 
that  there  are  characteristics  of  either  the  technique,  the  environment, 
or  both,  which  have  limited  the  applicability  and  usefulness  of  SADT  in 
the  TRADOC  context.  Data  from  other  sources  will  be  employed  in  other 
parts  of  this  report  to  further  illustrate  the  distinction  between  SADT's 
generic  qualities  as  exhibited  by  the  diagrams  themselves  and  the 
technique's  capacity  to  become  integrated  in  the  TRADOC  environment. 

C.  Qualitative  Data.  Discussion  questions  from  the  questionnaire 
and  a  follow-up  interview  with  respondents  shortly  after  the  administration 
of  the  questionnaire  provided  the  raw  data  for  the  following  interpretive 
comments.  Despite  differences  in  the  format  in  which  these  two  types  of 
data  were  collected,  considerable  consistency  among  verbal  and  written 
comments  emerged.  The  discussion  of  interpretive  data  begins  with  a 
presentation  of  the  most  general  conclusions  that  emerged  from  analysis 
of  the  subjective  comments  of  TRADOC  members. 

The  most  general  conclusion  shared  by  all  but  a  few  respondents 
at  the  end  of  Task  2  was  that  it  was  too  early  to  say  whether  the 
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technique  had  any  practical  value  for  TRADOC.  While  almost  all  respondents 
indicated  that  they  thought  the  technique  was  useful  at  a  general  context- 
free  level,  they  were  unsure  about  its  applicability  and  usefulness 
within  TRADOC.  In  response  to  the  first  discussion  item  on  the  question¬ 
naire  (Could  you  identify  any  specific  benefits  which  nay  have  accrued 
to  the  Aray  as  a  result  of  its  participation  in  this  project  thus  far?), 
the  following  consent  was  representative: 

Not  yet.  This  has  been  some  concern  to  ae 
since  the  project  started.  The  potential 
for  using  SADT  as  a  tool  to  improve  Aray 
training  is  real.  Perhaps  I  aa  not  involved 
enough . 

And,  in  a  follow-up  interview,  this  coament  was  expressed: 

We  went  into  this  project  to  get  someone  to 
provide  a  logical  layout  of  the  training 
system  so  we  could  get  a  handle  on  it.  Dumb 
grunts  like  myself  haven't  gotten  that  out  of 
it  yet.  I  don't  know  if  SofTech's  work  will 
be  of  any  use  at  this  time.... I'll  wait  before 
saying  yes  or  no. 


All  coamenters  could  readily  enumerate  strengths  of  SADT,  especially 
when  asked  about  the  technique  in  a  general,  context-free  sense.  While 
many  strengths  were  aentioned,  the  following  occurred  with  some  consistency: 

(1)  SADT  requires  a  graphic  presentation  of  the 
problen  which  simplifies  the  problem  and 
coamunicates  it  succinctly,  i.e,  a  picture 
is  worth  a  thousand  words; 

(2)  Commenting  forces  an  individuals  to  think 
critically  about  a  problem  before  he  can 
disagree  with  a  particular  diagram; 
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(3)  SADT  promotes  a  consensus  of  opinion  about 
a  previously  ill-defined  concept; 

(4)  SADT  saves  tine  over  verbal  communications 
because  it 4  a  hard  to  "get  off  the  subject" 
in  commenting  on  an  SADT  diagram; 

(5)  SADT  illuminates  the  real  causes  of  problems. 

Generally  speaking,  comment era  provided  positive  accounts  of  SADT;  all  but 
one  individual  expressed  that  they  believed  the  technique  was  a  good  one. 

When  asked  about  weaknesses  of  the  technique,  most  coamenters 
mentioned  aspects  of  the  application  of  SADT  and  not  generic  qualities  of 
the  diagrams  themselves.  The  most  consistently  mentioned  weaknesses  were 
that  the  SADT  process  was  too  time-consuming  and  that  commenting  on  a  kit 
occasionally  interferred  with  an  individual's  ongoing  work.  It  can  be 
seen  that  these  comments  mildly  contradict  those  made  about  the  strengths 
of  the  technique  (for  example,  #4  above).  This  paradox  was  evident  in 
many  of  the  comments  made  and  indicated  that  TRADOC  personnel  considered 
the  technique  during  this  phase  of  the  evaluation  as  a  good  one  generally, 
but  were  unsure  of  the  value  of  its  application  within  the  TRADOC 
context . 

Several  consistent  themes  and  comments  emerged  from  examination 
of  the  interpretive  data  which  help  illuminate  the  ambiguity  on  the  part 
of  TRADOC  personnel  about  the  value  and  usefulness  of  SADT  as  applied  to 
the  complex  problems  of  providing  Army  training.  One  of  the  most  consistent 
themes  was  that  insufficient  resources  had  been  expended  on  the  project 
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thus  far.  Item  43  of  cha  questionnaire  inquired  about  obstacles  which  may 
have  hindered  SofTech's  productivity  at  TRADOC.  The  following  response 
to  that  question  was  typical: 

The  normal  workload  af  DCST  (office  of  the  Deputy 
Chief  of  Staff  for  Training)  is  so  heavy  that 
numerous  key  personnel  were  not  able  to  get  in¬ 
volved  at  the  level  desired.  This  resulted  in 
reduced  visibility  of  the  project.  As  the  project 
grows  to  Task  3t  there  should  be  greater  involve¬ 
ment,  particularly  after  the  Coemander  Conferences 
(in  December).  TDY  (temporary  duty  involving  travel 
away  from  usual  assignment)  and  inaccessibility  of 
key  TRADOC  staff  —  this  may  have  hindered  them  in 
getting  the  job  done. 


Several  other  individuals  mentioned  that  they  had  not  devoted 
as  much  time  to  the  project  as  they  should  have.  This  may  be  one  reason 
why  individuals  valued  the  technique  generally  but  weie  unsure  of  its 
use  and  applicability. 

Another  consistent  theme  in  the  interprative  data  concerned  the 
command  emphasis  for  the  project.  Several  commenters  were  critical  of 
the  Army's  support  of  SofTech’s  work.  Much  of  this  criticism  was  aimed 
at  ’’upper  level  Army  management"  as  evidenced  by  the  following  comments 
from  interviewees  (all  below  the  rank  of  Colonel) : 


There  has  been  a  lack  of  command  emphasis.  IJve  had 
no  pressure  from  my  boss.  It  (participation  in 
SADT  project)  has  had  no  bearing  on  ay  job  as  far  as 
he  is  concerned. 

There  has  been  little  enthusiasm  for  this  project 
(SADT)  by  upper  level  management. 

Whether  it  will  be  used  here  is  strictly  dependent  on 
management. .. .If  managers  use  it  everyone  will  use  it; 
emphasis  has  got  to  come  from  the  top.  If  this  happens 
it  will  be  successful  here. 
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Anecdotal  coasMnts  written  in  the  margin*  of  mulitple-choice 
questions  indicated  that  usage  and  value  were  "entirely  dependent  upon 
coMwnd  influence,"  and  that  the  amount  of  tine  devoted  to  this  project 
"had  been  insufficient."  One  individual  indicated  that  all  questions 
pertaining  to  usage  were  dependent  on  management  and  he  doubted  that  they 
would  give  this  project  a  high  priority.  To  suaaurixe,  there  was  some 
skepticism  on  the  part  of  lower-ranking  officers  that  upper  level  Army 
management  was  actually  concerned  about  the  SADT  project. 

A  third  factor  contributing  to  skepticism  about  the  value  and 
usage  of  SADT  at  TRADOC  was  ambiguity  and  uncertainty  shared  by  most 
conmenters  of  what  the  project  results  should  be.  This  impression  becaam 
evident  during  inltal  contact  and  interviews  with  the  TRADOC  organisation. 
This  confusion  still  existed  at  the  conclusion  of  Task  2  as  indicated 
by  ths  following  comment  from  a  TRADOC  colonel: 

He  don't  know  what  our  (training)  system  is. 

He  don't  know  how  we  will  use  it  (the  SADT 
model  of  training).  I'm  unclear  in  my  own 
mind  about  how  I'm  going  to  use  it. .. .There's 
no  clear  perception  of  how  it  will  be  used 
by  me  or  anyone  I  know. 

A  fourth  consistency  evidenced  in  the  respondent's  comments 

concerned  the  timeliness  of  the  SADT  project.  The  project  closely  paralleled 

2  2 

the  Total  Tank  System  Study  (T  S  )  whose  goals  were  much  the  same  as  those 

of  Task  1  and  Task  2.  Several  respondents  pointed  out  that  SADT  was  im- 

2  2  2  2 

plemented  after  the  T  s*  study  was  well  under  way.  Because  T  S  and  the 
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SADT  project  overlapped,  both  In  terms  of  time,  course,  and  general  goals, 
the  relative  contributions  of  each  to  what  was  ultimately  learned  about 
tank  training  la  unclear. 

2  2 

The  chief  officer  at  the  T  S  group  had  the  most  to  say  on 

this  issue.  He  stated  that  the  T2S2  results  contained  the  "meat"  of  what 

was  learned  about  training,  but  that  the  work  done  using  SADT  geve  clearer 

definition  of  those  results.  He  felt  that  if  SADT  had  been  introduced  prior 

2  2 

to  the  beginning  of  the  T  S  study,  it  would  have  contributed  to  the  results. 

addressed  the  timing  issue  in  the  following  manner: 

1  was  immersed  in  real  problems;  we  didn't 
have  time  to  learn  new  techniques.  It  was 
initially  a  hindrance  tlmeswise  and  procedure 
ally. . .After  the  bulk  of  T ZSZ  was  finished 
we  had  time  for  SofTech  and  the  value  of  the 
technique  became  obvious.  2*$  conmunicates 
the  "big  picture"  of  the  T  SZ  results. 

Because  the  SADT  process  was  instituted  when  it  was,  and  given 

2  2 

that  its  goals  and  objectives  closely  paralleled  those  of  T  S  ,  the  value 
of  SADT  is  confounded  in  the  minds  of  most  TKADOC  officers.  To  paraphrase 
the  coaments  of  two  TRADOC  officers,  SADT  should  have  been  applied  to  a 
new  problem  that  had  never  been  studied  before  with  everyone  working  on  the 
problem  well-trained  in  SADT. 

Concerning  the  relative  contributions  of  each  study  to  the 

achievement  of  insights  into  tank  training,  the  consensus  was  that  SADT  had 

dv-ne  little  to  isolate  training  requirsmsnts  having  a  bearing  on  combat 

effectiveness  and  battlsfleld  readiness.  The  opinion  of  those  TSADOC  per- 

2  2 

sonnel  polled  was  that  the  T  S  group  actually  did  the  work  that  led  to 
conceptual  insights  with  differing  opinions  about  the  relative  contribution 
of  SADT. 
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Another  these  which  emerged  from  the  qualitative  dete  that  could 
account  for  the  skepticism  concerning  subsequent  usage  of  SADT  wee  that 
of  personnel  turbulence.  Several  respondents  mentioned  this  factor  as 
a  variable  moderating  the  acceptance,  usefulness  and  ultimate  success  of 
a  project  such  as  this  one.  One  officer  said  that  by  next  turner  perhaps 
as  many  as  five  of  the  top-ranking  officers  who  were  responsible  for  or 
participated  in  the  SADT  project  may  be  gone. 

It  is  apparent  that  personnel  turbulence  in  the  Army  will  effect 
most  any  project.  If  the  project  is  one  that  Involves  cooperation  and  time 
conmltments  from  particular  Army  personnel,  the  efforts  of  the  contractor 
are  particularly  dependent  on  personnel  changes  within  the  Aruy.  The 
longer  the  length  of  the  contract,  the  greater  the  probability  of  disruption 
to  the  ongoing  work.  The  problem  is  not  only  one  of  trained  Army  personnel 
being  lost  to  the  contractor  because  of  assignment  changes  but  also  one 
of  acquiring  the  time  of  new  officers  for  training  and  obtaining  the  needed 
commitment  from  a  commanding  officer  who  nay  also  be  new. 

A  final  theme  that  emerges  as  a  possible  obstacle  to  the  success¬ 
ful  implementation  and  usage  of  SADT  was  the  relative  difficulty  in  insti¬ 
tutionalizing  the  technique.  Host  individuals  polled  believed  that  for 
SADT  to  be  used  effectively  it  would  have  to  be  internalized.  With  few 
exceptions,  the  view  expressed  was  that  SADT  could  be  a  valuable  problesh- 
solvlng  technique  if  "brought  to  fruition  within  the  Arsry."  Opinions 
differed  on  how  to  internalize  the  technique  but  some  genetal  consistencies 
that  emerged  were: 
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(1)  it  takti  bright  individuals  to  learn  how  to 
author  SADT  diagrams,  not  everyone  could  be 
trained; 

(2)  if  used  by  operations  research  people,  then 
management  would  have  to  learn  and  accept  it 
in  order  to  coominicate  with  their  problem- 
solvers; 

(3)  if  accepted  and  used  by  management,  it  would 
filter  its  way  throughout  the  organisation; 

(4)  some  abridged,  simplified  form  that  did  not 
require  such  extensive  training  would  have  a 
better  chance  of  being  accepted  and  used; 

(5)  if  the  technique  clearly  solves  problems  and/or 
saves  time  and  manpower,  and  if  individuals 

are  sufficiently  trained,  the  technique  definitely 
will  be  used; 

(6)  complete  dependence  on  an  outside  contractor 
will  hinder  the  acceptance  of  any  technique. 

To  summarise,  although  most  respondents  (13  out  of  14)  indicated  that 
SADT  was  a  valuable  technique  at  a  general,  context-free  level,  there  was 
considerable  skepticism  about  its  subsequent  applicability  and  usefulness 
at  TRADOC.  Several  consistent  themes  that  emerged  fron  the  qualitative 

data  were  presented  as  potential  explanations  for  the  discrepancy  between 
the  context-free  and  context-specific  views  of  SADT.  Among  these  were: 

(1)  insufficient  resources  being  expended  on  the 
project; 

(2)  lack  of  coMand  emphasis  on  the  project; 

(3)  lack  of  clarity  about  project  goals; 

2  2 

(4)  overlap  between  the  SADT  project  and  the  T  S  group 
study; 

(5)  personnel  turbulence  in  the  Army; 

(6)  difficulties  in  institutionalising  a  technique 
such  as  SADT. 
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Discussion 


The  data  collactad  in  tha  Phaaa  1  • valuation  indicated  that 
officers  within  tha  TKADOC  organisation  conaldarad  SADT  a  powerful  and 
sophisticated  approach  to  systems  analysis,  design,  and  management. 
Questionnaire  iteas  relevant  to tha  context-free  value  of  tha  technique 
were  generally  rated  higher  than  other  items.  Consents  obtained  in  response 
to  discussion  questions  and  interview  probes  reinforced  the  conclusion  that 
the  generic  qualities  of  tha  diagrams  themselves,  such  as  communicative 
quality,  efficiency,  accuracy,  consistency,  and  completeness,  were  present 
as  claimed  in  the  SofTech  proposal. 

The  sane  is  true  about  SADT's  heuristic  properties.  When 
questioned  about  the  technique  par  ae,  most  individuals  indicated  a  belief 
in  SADT's  ability  to  promote  conceptual  insights  into  problems.  It  should 
be  mentioned  that  individuals  were  somewhat  less  sure  of  SADT's  heuristic 
qualities  than  they  were  about  the  generic  qualities  of  the  diagrams  them¬ 
selves.  It  can  be  concluded  that  the  sample  for  this  evaluation  study 
valued  the  SADT  methodology  and  generally  believed  the  claims  made  by 
SofTech  in  the  technical  proposal. 

Queries  about  the  value  of  SADT  to  the  TKADOC  organisation  did 
not  yield  the  same  positive  responses  as  did  questions  about  the  technique 
itself.  It  appears  that  sons  characteristics  of  the  SADT/TRADOC  interface 
moderated  individual's  opinions  of  SADT.  Both  quantitative  and  qualitative 
responses  about  SADT's  applicability  and  usefulnass  within  the  TKADOC 
context  ranged  from  mild  optimism  to  open  skepticism.  The  discrepancy 
between  context-free  and  context-specific  valuations  of  SADT  led  the 
evaluators  to  examine  more  closely  the  interface  between  the  technique 
and  the  environment.  The  process  data  collected  provided  a  starting  point 
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for  this  examination. 


The  available  process  data  indicated  that  SADT  had  not  been 
implemented  during  the  Phase  1  evaluation  to  the  degree  prescribed  in  the 
technical  proposal.  The  conclusion  drawn  from  these  data  is  that  partici¬ 
pation  by  TRADOC  personnel  in  the  iterative  author-conmenter  cycle  had 
been  approximately  half  that  expected  in  the  usual  application  of  SADT. 

The  limited  availability  of  TRADOC  personnel  for  authoring  SADT  diagrams 
#»•><*  the  fragmentation  of  the  TRADOC  interface  role  during  Task  2  further 
limited  the  implementation  of  SADT  at  TRADOC. 

The  open-ended  probes  built  into  the  evaluation  design  provided 
respondents  with  a  "free-hand"  in  expressing  their  views  about  the  SADT 
project,  and  from  these  interpretive  data,  potential  explanations  of  the 
marginal  implementation  of  the  technique  emerged.  The  most  consistently 
reported  of  these  were  that  insufficient  resources  had  been  expended  on  the 
project  and  that  there  had  been  a  lack  of  command  emphasis  for  the  project. 
Possible  explanations  for  these  project  weaknesses  Included  a  lack  of 
clr.rity  about  project  goals  (which  could  be  a  function  of  the  nature  of 
t'.ie  problems  addressed  by  this  project)  and  personnel  turbulence  within 
the  Army,  which  contributes  to  a  lack  of  continuity  in  management  perspective. 

Another  point  that  emerged  which  could  be  closely  related  to  a 
lack  of  command  emphasis  was  the  difficulty  encountered  in  attempting  to 
institutionalize  a  technique  such  as  SADT.  If  the  project  is  not  considered 
a  particularly  high  priority  by  Army  management,  this  is  probably  cowmi- 
cated  to  subordinates  in  many  ways,  e.g.,  officially  and/or  unofficially; 
purposefully  or  unintentionally.  If  communicated  to  subordinates,  difficulties 
will  probably  be  encountered  in  establishing  the  procedural  mechanics  of 
t'-?  project.  The  consent  of  one  officer  that  his  participation  in  the  SADT 


project  had  no  bearing  on  hla  job  aa  far  aa  hla  superior  was  concerned 
illustratea  thia  point. 

The  view*  of  Any  management  did  not  actually  refute  what  haa 
been  presented  thus  far.  These  individuals  admitted  that  So f Tech  had  not 
received  the  support  expected  and  offered  the  rationalisation  that  resource 
demands  always  exceed  resource  supply  in  the  TRADOC  environment.  One 

officer  stated  that  thia  resulted  in  TRADOC  being  an  unfair  teat  bed  for 

\ 

SADT.  Another  officer  stated  that  auch  a  hectic,  complex  environment  wes 
an  excellent  arena  in  which  to  aasess  SADT' a  applicability  to  complex 
nilitary  problems. 

The  appropriateness  of  the  testing  site  not  withstanding,  two 
conclusions  are  clear  from  examination  of  all  available  sources  of  Phase  1 
evaluation  data:  (1)  the  implementation  of  the  SADT  methodology  had  been 
marginal,  and  (2)  eight,  months  into  the  contract  most  TRADOC  personnel 
Involved  in  the  project  were  unsure  of  the  technique's  applicability  and 
usefulness  in  the  TRADOC  environment. 

In  answer  to  the  primary  questions  posed  for  the  Phase  1  evalu¬ 
ation  effort,  the  SADT  methodology  was  considered  to  be  a  highly  effective 
one  generally,  but  its  applicability  and  usefulness  within  the  TRADOC  environ¬ 
ment  was  moderated  by  several  factors.  Not  the  least  of  these  factors  were 
the  organisational  characteristics  of  the  adlitary,  which  predispose 
individuals  to  view  a  problem  one  way,  and  the  underlying  philosophy  of 
SADT,  which  prescribes  a  set  of  behaviors  for  attacking  a  problem  in  a 
•  •  •  'ferent  way. 
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VI.  Phase  2  Evaluation  Procedures  and  Rasul ta 


This  section  describes  the  procedures  and  results  of  the  Phase  2 
evaluation.  Using  the  Phase  1  evaluation  procedures  and  results  as 
guidelines,  the  Phase  2  evaluation  effort  was  concerned  primarily  with 
determining  whether  the  attitudes  of  TRADOC  personnel  towards  SofTech's 
work  had  changed  as  a  result  of  the  production  of  the  Task  3  model.* 
Emphasis  was  given  in  the  Phase  2  evaluation  to  the  perceived  effectiveness 
and  usefulness  of  the  Task  3  model,  how  this  model  was  being  used  and  what 
impact  the  model  was  having  on  Army  training  and  evaluation  concepts. 

This  phase  served  as  a  follow-up  to  the  Phase  1  evaluation,  up¬ 
dating  and  documenting  project  activities  and  describing  attitudes  of  TRADOC 
concerning  the  perceived  effectiveness  and  usefulness  of  SADT.  Examination 
of  current  project  activities  was  conducted  to  validate  solicited  opinions 
about  SADT  and  to  suggest  potential  indicators  of  the  ways  in  which  the 
SADT  project  may  impact  Army  training  and  evaluation  r-0grams. 

Methods  and  Procedures 

Mine  individuals  having  close  contact  with  the  SADT  project  were 
interviewed  approximately  one  month  after  the  completion  of  the  Task  3 
model  or  approximately  fourteen  months  after  the  start  of  the  project.  This 
«roup  consisted  of  five  colonels,  one  It.  colonel,  two  majors,  and  one 
captain.  The  conclusions  drawn  from  the  interview  data  are  presented 
below  in  order  of  their  consistency  across  the  nine  respondents.  Special 
weight  was  given  to  the  conents  of  those  having  major  responsibility  for 

*Task  3  resulted  in  the  generation  of  three  models  in  the  areas 
of  evaluation,  system  development,  and  training.  These  will  be  referred 
to  in  this  report  as  the  Task  3  model. 


this  project.  Appendix  D  contains  representative  stimulus  questions  used 
to  promote  discussion  during  the  Interview. 

The  interviewer  also  completed  a  structured  questionnaire  based 
upon  each  respondent's  interview  responses.  The  items  for  this  structured 
questionnaire  (which  appears  in  Appendix  C)  were  selected  from  among 
the  relevant  items  on  the  Phase  1  questionnaire  completed  by  these 
respondents  at  the  conclusion  of  Task  2.  This  permitted  the  comparison 
of  attitudes  before  and  after  the  completion  of  the  Task  3  model  for  a 
number  of  respondents. 

Results  and  Discussion 

The  most  consistent  theme  evident  from  the  comments  of  respondents 
was  that  the  Task  3  model  had  made  a  definite  contribution  in  bringing 
about  needed  changes  in  Army  training  and  evaluation  programs.  While 
respondents  varied  in  how  significant  they  thought  the  SofTech  contribution 
was,  it  was  clear  that  the  TRADOC  officers  polled  valued  the  Task  3  model 
and  considered  it  an  improvement  over  where  TRADOC  woujd  have  been  at  this 
point  without  it. 

Most  respondents  considered  the  major  value  of  SofTech' s  work  to 
be  in  documenting  the  interrelationships  between  evaluation,  system 
development,  and  training.  For  example,  the  Task  3  model  depicts  "inform¬ 
ative  feedback"  as  the  interface  between  the  heretofore  individually 
conceptualized  evaluation,  system  development,  and  training  components, 
thus  linking  these  functions  in  a  systematic  manner.  Respondents  indicated 
that  individuals  within  TRADOC  may  have  had  a  clear  conceptualization  of 
how  parts  of  the  system  for  which  they  were  responsible  operated,  but 
the  Task  3  model  provided  an  integrated  picture  of  the  total  Army 


training  and  evaluation  system  and  how  that  system  must  interact  with 
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combat  developments.  This  Insight  apparently  had  not  been  achieved,  or 
at  least  so  vividly  portrayed,  previously,  and  is  cited  here  as  the 
probable  basis  for  considerable  enthusiasm  by  TRADOC  personnel  for  actually 
beginning  the  work  of  implementing  the  new  training  and  evaluation  programs. 

The  following  comments  from  interviewees  are  representative  of 
this  newly  achieved  integrated  conception  of  evaluation,  system  design, 
and  training: 


The  SofTech  project  has  been  useful.  The  Army 
bureaucracy  is  filled  with  bright  fellows  working 
in  various  places  but  until  SofTech  came  these 
ideas  had  not  been  put  together.  SADT  was  a 
mechanism  for  bringing  about  interaction.  The 
ideas  -  were  there  all  along  but  had  not  been 
Integrated. 

Nothing  new  appeared  in  the  models  but  they  caused 
me  to  think  of  them  (training  Issues)  in  a  different 
light.  The  further  delineation  of  what  evaluation 
is  has  changed  our  concept  of  evaluation  and  how  it 
interrelates  with  the  training  system. . .This  year 
has  seen  the  realization  of  what  we've  been  (talking) 
about  for  three  years. 

The  discussion  with  SofTech  and  examination  of 
diagrams  was  one  of  several  Intellectual  activities 
that  defined  and  described  where  we  were  heading  in 
TRADOC.  The  Army  hasn't  taken  the  process  but  the 
products  of  that  process  are  valuable  in  where  we're 
heading. 

SofTech' s  work  has  made  people  think.  It  ties  things 
together.  SofTech 's  work  at  TRADOC  has  provided 
organization  to  TRADOC  itself;  it  has  provided  a 
sense  of  direction  tr  the  whole  concept  of  TRADOC. 

It  has  forced  TRADOC  to  a  clear  conceptualization  of 
what  they're  doing,  what  they  intend  and  what  they 
want  for  the  future. 

In  addition,  a  colonel  having  significant  responsibility  for  the 
SofTech  project  indicated: 

For  some  time  we  thought  we  understood  different  aspects 
of  the  system,  but  for  the  first  time  we  have  a  clear 
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picture  of  the  interrelationships  of  different 
functions  and  organizations  of  the  Amy  as  they 
relate  to  training.  The  model  systematized  and 
coalesced  our  knowledge. 

Between  the  Phase  1  evaluation  report  and  the  present  follow-up 
conducted  four  months  later,  a  marked  shift  in  TRADOC's  attitude  toward 
SADT  and  its  products  was  noted.  This  shift  was  traced  to  the  emergence 
of  the  Task  3  model  and/or  the  activity  which  surrounded  its  development. 

Most  of  the  skepticism  reported  by  TRADOC  personnel  in  the  Phase  1  report 
pertained  to  the  usefulness  of  SofTech's  work  and  the  ultimate  impact  it 
would  have  on  Army  training  and  evaluation  procedures.  At  the  end  of  Task  3, 
this  skepticism  had  been  replaced  by  enthusiasm  about  the  usefulness  of 
SofTech's  efforts.  The  Task  3  model  seemed  to  have  brought  together  and 
solidified  many  of  the  ideas  generated  by  TRADOC,  but  that  heretofore  were 
seen  as  disjointed  and  insufficiently  articulated  to  be  of  practical  use. 

The  conclusions  are  supported  by  results  from  the  structured  question¬ 
naire.  Several  items  had  higher  (more  positive)  mean  responses  after  Task  3 
than  they  did  at  the  conclusion  of  Task  2.  These  comparisons  are  presented 
in  Figure  5. 

Five  items  evidenced  relatively  large  changes  in  mean  response  across 
the  two  samplings.  The  largest  change  (of  about  1.5  scale  units)  occurred 
with  respect  to  the  respondent's  understanding  of  Army  training  and  evaluation 
programs  (item  5).  Other  changes  on  the  order  of  1  scale  unit  can  be  noted 
for  item  9,  indicating  that  respondents  were  now  more  confident  that  SADT 
diagrams  could  increase  TRADOC's  planning,  design  and  management  capabilities 
and  for  items  6,  7,  and  15  indicating  that  respondents  now  saw  more  uses 
for  the  models  produced,  saw  the  diagrams  as  more  instrumental  in  establishing 
training  requirements,  and  perceived  greater  impact  of  SADT  on  the  training 
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responses  to  selected  items  st  the  conclusion  of 
2  (dott*d)  at  the  conclusion  of  Task  3  (solid) 


concepts  currently  held  by  TRADOC.  Changes  between  .75  end  1  unit 
occurred  on  item  2,  10,  end  17,  suggesting  that  the  production  of  the 
Task  3  aodel  had  stiaulated  aore  coeeunicstion  seong  individuals  not 
directly  involved  in  the  project,  that  TRADOC  personnel  now  considered 
the  SADT  aodels  produced  aore  valuable  as  reference  docuaents  than  they 
did  at  the  conclusion  of  Task  2  and  that  TRADOC  personnel  ware  now  aore 
convinced  that  the  resources  devoted  to  this  project  had  been  well-spent. 

The  Phase  1  evaluation  indicated  that  the  degree  of  lapleawntation 
of  the  SADT  methodology  had  been  aarglnal,  particularly  the  training  of 
SADT  authors,  and  suggested  that  lack  of  implementation  night  Halt  the 
utility  and  iapact  of  SofTech's  work.  The  skepticism  about  the  value  of 
the  work  produced  through  Task  2  was  attributed  earlier  to  this  lack  of 
implementation.  It  may  be,  however,  that  the  content  of  the  Task  1  and 
Task  2  aodels  limited  the  statements  that  could  have  been  made  about  ulti¬ 
mate  usage  and  impact.  Thus,  the  actual  production  of  the  elaborated  aodel 
of  Army  training  and  evaluation  resulting  from  Task  3  may  have  been  a 
necessary  prerequisite  to  TRADOC  officers  verbalizing  any  specific  benefits 
of  the  SADT  project.  Further,  positive  valuations  of  SofTech's  work  at  the 
end  of  Task  3  suggested  that  the  degree  of  participation  by  TRADOC  in  the 
SADT  process  had  been  sufficient,  although  less  than  originally  expected. 

Tasks  1  and  2  represented  necessary  but  preliminary  steps  that 
prepared  the  SADT  analysts  for  the  Task  3  effort.  One  officer  stated  that 
SofTech  "cut  their  teeth"  on  Tasks  1  and  2.  During  this  time,  officers 
had  a  difficult  time  seeing  the  value  of  SofTech's  work.  Because  no  one 
could  articulate  what  a  aodel  of  the  Army  training  system  should  look  like 
before  actual  production  of  that  aodel,  TRADOC  officers  may  have  been 
uncertain  as  to  how  the  Task  1  and  2  results  would  provide  a  better 
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understanding  of  Any  training  and  evaluation  program.  This  also  suggests 
that  project  benefita  to  TRADOC  are  directly  attributable  to  the  tangible 
outputs  of  the  SADT  process. 

In  sumary.  attitudes  toward  SofTech's  efforts  at  the  conclusion 
of  Task  3  were  positive.  This  represents  a  shift  froai  generally  neutral 
or  skeptical  attitudes  expressed  by  TtADOC  personnel  Interviewed  at  the 
end  of  Task  2.  The  relationship  between  training,  evaluation,  and  systaa 
development  was  perceived  by  TtADOC  personnel  to  be  especially  clear  at 
the  completion  of  the  Task  3  nodal,  while  the  United  focua  of  Tasks  1 
and  2  nay  have  contributed  to  the  skepticlsa  and  ambiguity  noted  in  the 
Phase  1  evaluation. 

Although  there  was  a  high  degree  of  enthusiasm  at  TtADOC  about  the 
new  conceptualisation  of  Any  training  and  evaluation,  it  would  te  pre¬ 
nature  to  conclude  that  this  result  was  solely  attributable  to  the  pro¬ 
duction  of  the  Task  3  model.  Other  factors  contributing  to  this  outcome 
may  have  Included  the  particular  insights  (and  foresights)  of  key  TRADOC 
officers,  the  activities  of  other  projects  at  TRADOC,  subtle  organisa¬ 
tional  characteristics  which  may  have  evolved  at  TRADOC,  and  the  seitgelst 
currently  present  In  TRADOC  and  in  the  Army.  While  this  list  is  specu¬ 
lative  and  far  from  inclusive,  it  is  offered  to  indicate  that  TRADOC' a 
current  understanding  of  training  and  evaluation  may  be  the  result  of 
the  confluence  of  a  large  number  of  variables,  many  of  which  could  not 
be  masured  given  the  time  and  resources  allotted  to  this  study.  Further¬ 
more,  these  potential  influences  may  interact  with  each  other  in  complex 
ways  resulting  in  unique  combinatorial  effects. 
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Thus,  a  pot ant tally  large  array  of  Influences  aay  have  interacted 
to  produce  the  attitudes  Measured  in  the  Phase  2  evaluation.  The  opinion 
expressed  by  aost  TRADOC  personnel  was  that  SofTech'a  input  was  a  part 
of  this  larger  confluence.  Two  officers,  however,  expressed  that  SADT 
brought  organisation  to  TRADOC,  that  it  created  a  different  clieate  at 
TRA33C,  Influenced  other  projects,  etc.  This  would  suggest  that  this 
combine  of  Influences  falls  within  the  Influence  of  SADT  itself.  The 
representations  in  Figure  6  portray  several  of  the  conditions  that  aay 
have,  existed  at  TRADOC  to  produce  the  attitudes  measured  during  the 
Phase  2  evaluation. 


Figure  6 

Combination  of  all  influences 
affecting  TRADOC* 8  perception  of  Army  training 


SADT  is  among  many 
factors  influencing 
TRADOC '8  perception 
of  Army  training 


SADT  is  the  only 
factor  influencing 
TRADOC 's  perception 
of  Army  training 


SADT  is  the  primary 
factor  influencing 
TRADOC' s  perception 
of  Army  training. 


Which  of  these  relationships  aost  accurately  portrays  the  role  of  SADT 
in  fostering  change  at  TRADOC  Headquarters  cannot  be  determined  by  the 
present  data.  This  discussion,  then,  is  presented  as  a  caution  to  those 
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who  would  Interpret  the  Phase  2  evaluation  as  indicating  that  the 
production  of  the  Task  3  aodel  was  the  sole  causal  agent  in  bringing 

about  changes  in  Army  training  and  evaluation. 

That  TRADOC  was  pleased  with  the  product  of  the  project  at  the 

conclusion  of  Task  3  was  evident.  The  following  consent  from  a  TRADOC 
colonel  sumsmrises  what  TRADOC  received  froa  this  contract  through 

Tar*.  3: 

The  principle  value  (of  this  project)  has 
been  the  identification  of  sources  of  Infor¬ 
mation  relative  to  how  the  ayatea  is  working 
and  feeding  that  information  back  into  the 
front  end  (systea  development) .  (This)  will 
Increase  our  ability  to  better  target  resources 
we  put  into  training  and  hardware  developaent. 

The  contract  proposal  listed  three  specific  benefits  that  would  , 
be  derived  from  the  Task  3  model: 

(1)  better  understanding  in  TRADOC  of  what  it  is  trying  to 
achieve ; 

(2)  ease  of  telling  others,  the  schools,  and  Departaent  of 
the  A ray  what  is  going  to  happen;  and 

(3)  reduction  of  tine  in  planning,  sumaging,  and  accomplishing 
the  changes.  (Contract  proposal  p.  3-10)  . 

At  the  conclusion  of  the  Phase  2  evaluation  these  benefits  had 


occurred . 
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VII.  Phase  3  Evaluation  Procedures  and  Results 


General  Approach 

This  section  describee  the  procedures  and  results  of  the  Phase  3 
evaluation.  Building  on  the  results  of  the  Phase  1  and  Phase  2  evaluations. 
Phase  3  determined  post-project  opinions  of  TRADOC  personnel  towards  SADT. 

The  focus  of  Phase  3  differed  somewhat  from  that  which  was 
originally  planned.  Because  this  change  necessitated  a  shift  in  evaluation 
strategy,  a  discussion  of  the  rationale  for  this  change  follows. 

Although  the  Task  3  report  was  well-accepted  at  TRADOC  Headquarters, 
the  Task  3  models  had  received  little  exposure  at  the  21  TRADOC  schools 
at  the  conclusion  of  Task  3.  The  models  were  considered  reasonably  specific 
by  personnel  at  TRADOC  Headquarters  but  the  viewpoint  of  individuals  at  the 
TRADOC  schools  was  that  the  models  were  abstract  and  of  questionable  practical 
value.  Further,  the  models  emphasised  new  roles  for  the  schools  involving 
heavy  usage  of  exportable  training,  assigning  various  schools  the  responsi¬ 
bility  for  weapon  system  performance  and  requiring  schools  to  use  performance- 
oriented,  criterion-referenced  measures  of  training  effectiveness.  The 
traditional  role  of  TRADOC  schools  (training  resident  students)  was 
deemphasized  in  the  Task  3  model.  Following  dissemination  of  the  Task  3 
report  it  did  not  appear  likely  that  TRADOC  school  personnel  would  readily 
adopt  the  new  model  of  training  because  of  its  lack  of  specifics  about 
each  school *8  weapon  system  and  MOSs  and  the  emphasis  on  newer  rsther 
than  more  traditional  roles  for  TRADOC  schools. 

It  was  evident  to  SofTech  and  TRADOC  personnel  at  the  conclusion 
of  Task  3  that  acceptance  of  the  new  conceptualisation  of  Army  training 
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and  evaluation  at  the  various  proponent  schools  vas  critlcsl  to  the 
successful  Implementation  of  the  model.  Further,  because  of  the  extensive¬ 
ness  of  the  Tssk  3  model.  It  was  believed  that  emphasis  should  be  concen¬ 
trated  on  TRADOC's  most  imsMdlate  critical  needs  rather  than  the  broad 
focus  of  developing  an  implementation  plan  for  the  entire  Task  3  model. 

For  these  reasons,  the  decision  was  made  to  drop  the  implementation 
plan  which  was  the  original  focus  of  Task  4.  Instead,  effort  was  to  be 
devoted  in  Task  4  to  several  activities,  including  dissemination  of  the  new 
model  of  Army  training  to  the  proponent  schools  of  TRADOC  to  foster 
acceptance  of  the  model,  development  of  a  plan  for  implementing  some  specific 
evaluation  feedback  loops  into  the  existing  training  structure,  and 
beginning  a  plan  for  a  system  architecture  for  the  data  processing  support 
requirements  of  the  new  model  of  training. 

Because  the  new  focus  of  Task  4  did  not  emphasize  the  use  of  SADT, 
the  Task  4  activities  were  not  as  directly  relevant  to  the  purpose  of  the 
overall  evaluation — the  evaluation  of  SADT  in  modeling  a  military  training 
environment .  Further,  these  and  other  project  activities  subsequent  to 
Task  3  did  not  occur  early  enough  to  allow  for  an  analysis  prior  to  the 
preparation  of  this  report.  For  these  reasons  the  Phase  3  evaluation  focused 
on  a  finer  determination  of  TRADOC's  perceptions  of  the  effectiveness  and 
usefulness  of  the  SADT  project,  particularly  the  Task  3  model  and  specifically 
the  factors  which  may  have  been  responsible  for  the  positive  shift  in  attitude 
which  occurred  between  the  Phase  1  and  Phase  2  evaluations. 

Methods  and  Procedures 

The  data  collection  period  for  the  Phase  3  evaluation  occurred 
during  July  and  early  August  1977.  The  personal  vacation  schedules 
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and  professional  obligations  of  TRADOC  personnel  during  this  tins  period 
made  it  impractical  to  atteapt  to  interview  relevant  TRADOC  officers  in 
depth  about  their  perceptions  of  the  SADT  project  and  the  Task  3  nodal. 

For  these  reasons  a  nail-out  questionnaire,  siisilar  to  the  one  used  in 
the  Phase  1  evaluation,  was  selected  as  the  primary  data  source  for 
Phase  3.  This  questionnaire  appears  in  Appendix  D. 

Fifteen  TRADOC  officers  having  management  responsibility  for  or 
participating  in  the  SADT  project  were  selected  as  the  target  population 
for  the  questionnaire.  Two  weeks  after  the  questionnaire  was  nailed, 
an  evaluation  staff  nenber  went  to  Ft.  honroe  to  stinulate  the  return 
rate  of  questionnaires  and  to  interview  as  many  of  the  target  population 
as  possible  about  their  overall  impressions  of  the  project.  Questionnaire 
and  interview  results  are  presented  and  discussed  below. 

Results  and  Discussion 

Twelve  of  the  fifteen  questionnaires  were  returned,  a  return 
rate  of  80Z.  Further,  four  Individuals  were  interviewed  during  a  final 
visit  to  TRADOC  Headquarters  by  an  evaluation  staff  member.  The  results 
of  these  data  gathering  activities  yielded  two  types  of  data,  quantitative 
(from  Questions  1  through  21  of  the  questionnaire,  Appendix  E)  and 
qualitative  (from  questions  22  through  27  of  the  questionnlare  and 
the  interviews) . 

Quantitative  Data.  A  reliability  coefficient  was  calculated 
for  the  21  quantitative  items  on  the  questionnaire.  An  alpha  coefficient 
of  .93  was  obtained  indicating  that  questionnaire  items  were  homogeneous. 
That  is,  they  produced  similar  patterns  of  responding  in  different  indi¬ 
viduals,  indicating  the  questionnaire  was  reliable. 
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The  mean  (X)  end  standard  deviation  (SD)  for  each  item 
are  presented  in  Table  4. 

Inspection  of  the  general  intent  of  each  itea  suggests  that  lteas 
can  be  clustered  into  relatively  homogeneous  groups.  Por  example,  lteas 
3,  4,  7,  9,  13,  and  14  Inquire  about  various  heuristic  aspects  of  SADT 
diagrams  and  the  modeling  process.  Inspection  of  the  results  of  these 
questions  suggests  most  notably  that  the  SADT  models  produced  were  helpful 
in  elucidating  the  interrelationships  between  the  various  components  of 
training  (Table  4,  Item  4)  and  that  the  Task  3  model  provided  a  clearer 
understanding  of  the  Army's  training  and  evaluation  programs  (Itea  7). 
Responses  to  Items  13  and  14  suggested  that  the  technique  was  somewhat 
less  effective  in  identifying  organisational  inefficiencies  in  the 
current  training  system  or  in  providing  conceptual  insights  about  alterna¬ 
tive  methods  of  conducting  Army  training.  None  of  the  questions  addressing 
heuristic  properties  of  the  technique  received  unfavorable  ratings 
(i.e. ,  a  mean  less  than  3.00  on  the  five-point  scale). 

Items  8,  12,  and  IS  address  the  ultimate  usage  and  Impact  of 
the  models  produced.  While  respondents  believed  the  models  would  be 
used  to  a  considerable  extent  in  planning  needed  changes  in  Army  training 
(Item  8) ,  they  were  relatively  less  sure  that  the  models  would  actually 
provide  the  impetus  for  change  (Item  12)  or  that  the  project  would  ulti¬ 
mately  affect  the  training  concepts  held  by  TRADOC.  As  with  the  previous 
grouping  of  items,  none  of  these  questions  received  unfavorable  ratings. 

Several  items  on  the  questionnaire  (10,  11,  16,  17,  19,  20  and 
21)  asked  the  respondent  for  his  summary  Judgments  about  various  aspects 
of  the  SADT  project.  The  most  notable  of  these  was  that  Item  11  received 
the  highest  ratings  of  any  item  on  the  questionnaire  (4.55),  indicating 
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1.  Are  SADT  diagrams  an  effective 


way  of  communicating? 

12 

4.33 

0.6S 

2. 

Has  SADT  project  generated 
communication  among  TRADOC 
personnel? 

12 

3.75 

0.87 

3. 

Have  diagrams  helped  focus 
attention  on  variables  not 
considered  previously? 

12 

3.67 

1.23 

4. 

Did  models  elucidate  inter¬ 
relationships  between  the 
components  of  training? 

12 

4.25 

0.75 

5. 

Project  benefits  attributable 
to  diagrams  or  process? 

11 

3.00 

0.89 

6. 

Has  SADT  saved  time  relative 
to  other  approaches? 

10 

2.80 

1.32 

7. 

Did  Task  3  model  provide 
clearer  understanding  of 

Army  training? 

11 

4.00 

0.89 

8. 

Will  the  models  produced  be 
used  in  planning  changes? 

12 

3.50 

0.80 

9. 

Has  SADT  helped  identify 
changes  Affecting  combat 
readiness? 

12 

3.50 

1.45 

10. 

Would  an  SADT  model  of  Army 
training  be  of  value  as  a 
reference  document? 

12 

4.33 

0.89 

11. 

Is  SADT  an  effective  approach 
to  design  and  analysis  problems? 

11 

4.55 

0.69 

12. 

Will  SADT  models  provide  impetus 
for  changes  in  Army  training? 

12 

3.17 

1.03 

13. 

Were  SADT  models  useful  in  iden¬ 
tifying  existing  organ! rational 
inefficiencies? 

12 

3.17 

0.94 

75 


Table  4  (continued) 


Item 


Standard 

N  Mean  Deviation 


14.  Had  SADT  led  to  conceptual 
insights  about  conducting 


Army  training? 

11 

3.45 

1.21 

15. 

Will  use  of  SADT  ultimately 
affect  training  concepts 
held  by  TRADOC? 

12 

3.25 

0.75 

16. 

Would  it  be  useful  to  train 
some  Army  personnel  to  be 

SADT  authors? 

12 

3.67 

1.15 

17. 

Has  the  project  increased 

TRADOC' s  analytic  and 
planning  capabilities? 

12 

3.17 

0.83 

19. 

Has  the  progress  made  been 
worth  the  time  spent? 

12 

4.42 

0.79 

20. 

Estimate  of  SofTech's 
contribution  to  the  progress 
made  by  TRADOC. 

12 

3.08 

0.79 

21. 

Task  3  model  an  Improvement 
over  previous  conceptuali¬ 
zations  of  training? 

11 

4.36 

1.12 

Overall 


3.67 


0.55 


a  strong  belief  among  TRADOC  personnel  that  SAOT  is  an  effective  and  appro¬ 
priate  approach  to  the  design  and  analysis  problems  faced  by  TRADOC.  These 

individuals  co  lsldered  the  Task  3  model  an  improvement  over  previous  con- 
ceptualizatl  /ns  of  training  (Item  21)  and  were  of  the  opinion  that  the 
progress  '^ade  was  worth  the  resources  devoted  to  the  project  (Item  19). 
”~..<.ner,  these  individuals  considered  the  model  to  be  a  valuable  future 
reference  document  for  TRADOC  personnel  (Item  10) . 

Despite  the  high  ratings  given  to  most  items.  Question  17  was 
marked  relatively  lower  (X  -  3.17)  suggesting  that  a  limited  degree  of 
technology  transfer  had  occurred.  This  is  congruent  with  conclusions 
drawn  from  the  process  data  in  the  Phase  1  evaluation  where  it  was 
reported  that  fewer  than  expected  TRADOC  personnel  were  trained  in 
SADT  and  less  than  full  participation  among  TRADOC  personnel  was  recorded 
for  the  author/commenter  review  cycle.  Item  20  also  received  relatively 
low  ratings  (X=  3.08),  suggesting  that  while  SofTech  made  a  moderate 
contribution  to  the  progress  achieved  by  TRADOC,  it  was  not  a  major 
contributor.  This  might  be  expected  from  the  interaction  of  SADT  and 
other  influences  on  Army  progress  discussed  at  the  conclusion  of  the 
Phase  2  evaluation  results  (pp.  63-64  of  this  report). 

Qualitative  Data.  The  questionnaire  contained  six  open-ended 
questions  (Appendix  D,  Questions  22,  23,  24,  25,  26,  and  27).  One  of 
these  (26)  will  be  discussed  in  a  subsequent  section  of  this  report. 
Questions  22,  23,  24,  and  25  addressed  specific  issues  that  were  either 
left  unclear  or  emerged  from  the  Phase  1  and  2  evaluations. 

A  conclusion  drawn  from  the  Phase  2  evaluation  was  that 
attitudes  at  TRADOC  toward  SADT  had  shifted  positively  following 
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dissemination  of  the  Task  3  report.  Several  tentative  explanations  for 
this  shift  were  offered  at  the  conclusion  of  the  Phase  2  evaluation 
report.  The  opinions  of  TRADOC  personnel  were  solicited  to  further  specify 
why  this  shift  had  occurred  (Question  22).  The  most  consistent  response 
to  this  question  was  that  Tasks  1  and  2  were  necessary  preliminary 
activities  of  limited  scope  while  the  Task  3  model  constituted  the  "meat" 
of  the  project.  The  following  comments  by  four  officers  are  representative: 


Since  Task  2  dealt  with  a  specific  weapons  system 
(the  tank),  many  officers  were  either  unfamiliar 
or  uninterested  in  the  Task  2  report.  The  general 
subject  of  the  training  system  is  more  widely 
understood  and  generated  a  greater  degree  of  interest. 
The  meat  of  the  contract  was  in  Task  3  and  Tasks  1 
and  2  were  merely  preliminaries. 

Task  3  seemed  to  come  to  grips  better  with  the 
key  issues.  After  all,  it  represented  a  high 
point  in  the  learning  curve.  Simultaneously 
it  occurred  as  we  were  trying  to  deal  with 
(other  agencies)  on  total  system  developments. 

Hence,  its  insights  were  of  benefit  in  the 
daily  battle. 

The  Task  2  report  was  not  promulgated  as  a 
solution.  It  was  a  means  to  an  end.  The  Task  3 
report,  on  the  other  hand,  was  more  general 
and  intended  for  wider  distribution.  Thus, 

Improvement  in  opinion  should  occur. 


Task  3  showed  the  "big  picture"  for  the 
first  time.  People  could  see  how  their 
piece  fit  in  and  where  the  gaps  were. 


Question  2A  attempted  to  assess  whether  the  SADT  project 
had  had  an  impact  on  TRADOC's  ability  to  solve  large  and/or  complex  problems. 
While  this  was  one  of  the  major  benefits  implied  in  the  contract  proposal. 
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Che  issue  had  not  been  addressed  directly  in  Che  Phase  1  and  Phase  2 
evaluations.  The  general  response  to  this  question  was  that  the 
i  Jividual  officers  directly  involved  in  the  project  had  probably  honed 
their  personal  problem-solving  abilities  but  that  SADT  had  not  had  an 
organization-wide  impact  on  analysis  and  planning  skills.  It  should  be 
mentioned  that  several  respondents  believed  that  a  lack  of  command 
emphasis  during  the  project  may  have  limited  the  impact  the  project 
could  be  expected  to  have  on  these  skills. 

A  major  conclusion  of  the  Phase  2  evaluation  report  was  that 
following  dissemination  of  the  Task  3  report,  individuals  were  more 
confident  of  the  usefulness  of  the  SADT  project  and  were  more  positive 
that  this  Work  would  ultimately  be  of  benefit  to  TRADOC.  However,  it 
was  unclear  from  the  Phase  2  data  just  how  the  Task  3  model  would  be 
used  or  what  specific  benefits  have  or  were  expected  to  accrue  at 
TRADOC. 

Question  23  inquired  directly  about  ways  the  Task  3  model  would 
be  used  by  TRADOC.  Emphasis  was  given  to  the  responses  of  higher  ranking 
officers  with  training  management  responsibility.  Responses  to  this 
question  fall  into  three  general  categories. 

The  most  consistently  mentioned  usage  of  the  model  was  as  a 
communications  tool.  One  officer  stated  that  the  model  would  be  used 
to  "clarify  the  DA  (Department  of  the  Army)  staff's  understanding  of 
the  training  system."  Another  officer  stated  that  a  major  use  of  the 
model  was  "to  explain  the  pervasiveness  of  TRADOC' s  functions  and  missions 
to  DA,  DoD,  and  other  major  commands."  Other  responses  suggested  Task  3's 
usefulness  as  a  communications  tool  within  TRADOC,  stressing  that  the 
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model  can  be  used  to  explain  to  TRADOC  personnel  the  Interrelationships 
between  TRADOC  and  other  coonands  and  agencies  within  the  Army  and  the 
DoD.  It  is  clear  that  a  major  usage  of  Task  3  is  as  "a  way  to  explain 
the  training  system." 

Another  consistently  noted  usage  of  the  Task  3  model  was  as  an 
aid  in  the  systematization  and  coordination  of  combat  development  and 
ti dining  development  activities.  This  was  discussed  as  a  major  conclusion 
of  the  Phase  2  evaluation  and  will  not  be  elaborated  here.  It  should 
suffice  to  say  that  the  role  of  the  Task  3  model  as  a  blueprint  for  the 
coordination  of  combat  development  efforts  (primarily  the  development 
of  specific  weapons  systems)  and  training  developments  efforts  continues 
as  one  of  the  most  valued  of  the  potential  uses  of  the  Task  3  model. 

A  third  category  of  uses  centers  around  the  value  of  the  Task  3 
model  as  an  analytical  tool  in  such  areas  as  the  planning  of  management 
information  requirements  for  the  training  system  and  determining  the 
system  architecture  necessary  to  support  future  data  processing  needs. 
Additional  usages  within  this  area  include  providing  management  a 
perspective  for  assigning  responsibility  for  various  aspects  of  the 
training  and  evaluation  system  and  establishing  criteria  for  the  alloca¬ 
tion  of  training  resources. 

At  the  conclusion  of  Task  3,  most  respondents  interviewed  for 
the  Phase  2  evaluation  report  expressed  that  it  was  too  early  to  specify 
benefits  that  might  accrue  to  TRADOC  as  a  result  of  carrying  out  this 
project.  The  Phase  3  evaluation  questionnaire  followed  these  interviews 
by  approximately  three  months,  allowing  time  for  the  model  to  "sink  in" 
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and  for  potential  benefits  of  the  model  to  evidence  themselves.  As 
with  the  uses  of  the  Task  3  model  discussed  above,  several  specific 
benefits  were  articulated  by  respondents  In  response  to  Question  25. 

A  primary  benefit  listed  by  almost  all  respondents  was  that  the 
SofTech  project  had  resulted  In  the  articulation  of  the  Army  training 
and  evaluation  system  for  the  first  time.  This  documentation  was  seen 
as  very  valuable  by  TRADOC  personnel  and  Is  related  to  their  perceived 
usage  of  the  Task  3  model  as  a  valuable  coomunications  tool.  Another 
often  cited  benefit  was  the  elucidation  of  the  relationship  between 
combat  developments  and  training  developments.  A  third  consistently 
occurring  response  to  Question  25  was  that  "the  role  of  evaluation  and 
feedback  in  the  training  system  had  been  clearly  identified  and  raised 
to  the  desired  level  of  prominence." 

Other,  more  idiosyncratic  benefits  listed  in  response  to  Item  25, 
were  that  because  SADT  highlights  system  constraints,  required  "fixes" 
tc  present  training  system  problems  could  now  be  more  easily  identified. 
And,  because  the  Task  3  model  "highlighted  areas  where  automation  was 
n.oded,"  a  starting  point  for  planning  future  hardware/software  needs 
could  be  determined. 

A  different  perspective  was  offered  by  one  officer  who  listed 

the  only  project  benefit  as  the  "crossfertilisation  of  ideas."  He  ex- 

nlalned  his  response  this  way: 

(The  project)  forced  TRADOC  to  explain  the 
system  used  to  outside  agents.  This  is  a 
remarkably  useful  exercise  for  smug  persons 
who  think  they  understand  what  they  do. 
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Given  the  evaluator’s  experience  with  SADT  and  TRADOC,  this  coatnt  aeeas 
Insightful  and  nondefensive,  and  appears  to  underscore  the  smchanics  of 
how  SADT  accomplishes  its  results. 

The  idiosyncratic  comments  of  another  TRADOC  officer  deserve 

mention.  While  acknowledging  the  potential  benefits  of  SADT,  he  had 

reservations  about  whether  any  benefits  would  actually  be  realised.  His 

response  to  Item  25  follows: 

As  a  minimum,  TRADOC  has  been  exposed  to  an 
enlightened  form  of  analysis.  The  results 
now  exist  in  a  highly  usable  form.  I  am  con¬ 
cerned  about  TRADOC 's  ability  to  apply  the 
Task  3  model.  This  concern  is  based  upon  our 
lack  of  experience  in  conducting  such  a  massive 
project — the  type  of  project  a  policy-oriented 
staff  is  not  equipped  to  handle. 

Thus,  the  benefits  of  the  model  as  documentation  of  what  TRADOC 
is  trying  to  achieve  is  clear  in  the  minds  of  most  TRADOC  officers.  Even 
the  process  of  working  through  the  models  was  considered  to  be  of  benefit 
tn  the  organization  in  and  of  Itself.  Further,  the  model  appears  to  have 
considerable  potential  value  as  TRADOC  Headquarters  moves  to  make  needed 
changes  in  the  current  Army  training  and  evaluation  system.  There  is 
some  skepticism,  however,  that  TRADOC  will  be  able  to  use  the  model  opti¬ 
mally.  As  stated  by  one  officer  "the  value  of  SADT  is  still  more  'potential' 
than  realized." 

Summary  and  discussion.  The  most  notable  trend  from  the  Phase  3 
evaluation  is  that  the  attitudes  of  TRADOC  personnel  towards  the  Task  3 
model  reuain  positive.  No  deterioration  of  the  positive  valuation  of 
the  SADT  work  was  evident  three  months  after  dissemination  of  the 
Task  3  report.  Further,  it  appears  that  this  three  month  interval  pro¬ 
vided  an  "incubation  period"  in  which  the  results  of  the  Task  3  work 
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had  a  chance  to  "sink  In"  in  the  minds  of  TRADOC  personnel.  Phase  3 
evaluation  results  indicated  the  model  had  not  been  put  on  a  shelf 
and  forgotten  but  rather  had  become  a  central  document  in  TRADOC 'a 
thinking  about  training  and  evaluation.  The  high  return  rate  for  the 
Phase  3  questionnaires  and  the  thoroughness  with  which  most  respondents 
answered  items  on  the  questionnaire  seems  to  support  this  conclusion. 

The  conclusions  drawn  from  the  Phase  3  evaluation  data  were 
similar  to  those  in  Phase  2,  namely  that  TRADOC  is  satisfied  with  the 
results  and  considered  its  investment  in  the  project  worthwhile.  The 
general  opinion  expressed  was  that  the  groundwork  had  been  laid  to  con¬ 
vert  the  Army  training  system  from  what  it  is  to  what  it  should  be. 
Whether  the  potential  benefits  of  the  SofTech  project  are  realised  is 
dependent  on  two  factors  in  the  minds  of  most  TRADOC  personnel.  The 
first  concerns  the  emphasis  given  to  the  model  as  a  planning  tool  by 
upper  level  management  at  TRADOC.  Throughout  all  three  evaluation 
phases,  this  has  been  an  expressed  concern  of  officers  involved  in  the 
project.  The  ultimate  use  (and  derived  benefit)  of  the  model  is  depen¬ 
dent  on  the  priority  given  it  by  senior  level  managers  in  TRADOC.  The 
second  factor  concerns  reservations  about  the  ability  of  TRADOC  (more 
specifically  ODCST)  to  use  the  results  of  the  Task  3  model  in  working 
toward  the  state  of  training  and  evaluation  desired  by  TRADOC. 

The  interrelationships  between  the  results  of  the  three  eval¬ 
uation  phases  and  a  final  project-wide  evaluation  sutasary  follows. 
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VIII.  Project  Results  and  Discussion 


In  this  section  trends  across  the  three  evaluation  phases  are 
integrated  and  discussed. 

Process  Data 

Considerable  emphasis  during  the  Phase  1  evaluation  was  placed 
on  the  SADT  process.  With  the  exception  of  one  TRADOC  officer  trained 
as  an  SADT  author,  commenting  on  and  approving  SADT  kits  represented  TRADOC* 
only  official  contact  with  SADT.  Thus,  any  benefits  derived  by  the  client 
organization  would  seem  dependent  on  completion  of  these  process  behaviors. 
It  was  for  this  reason  that  the  process  data  from  the  project  file  was 
examined  during  Tasks  1  and  2. 

The  Phase  1  evaluation  report  indicated  that  TRADOC *s  participa¬ 
tion  in  the  process  of  diagram  review  and  revision  was  about  50X  of  that 
originally  expected  by  SofTech.  Despite  this  result,  none  of  the  project 
personnel  polled  at  the  conclusion  of  Task  2  believed  that  deficiencies 
perceived  in  the  project  at  that  time  were  attributable  to  this  reduced 
degree  of  participation  in  the  iterative  author-consnenter  cycle.  Most 
respondents  indicated  that  ample  conmunicatlon  had  occurred  between  SADT 
authors  and  TRADOC  commenters  and  that,  if  anything,  the  formality  of  the 
SADT  process  had  hindered  TRADOC' s  participation  in  the  project  to  some 
extent. 

Prior  to  completion  of  Task  2,  SofTech  altered  the  SADT  process 
slightly  to  achieve  a  better  match  between  the  process  and  work  procedures 
in  the  TRADOC  organization.  Primarily,  communication  within  the  SADT 
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process  was  expanded  to  Include  conferences  and  open  discussion  of  the 
status  of  various  aspects  of  the  model.  Because  both  SofTech  and  TRADOC 
expressed  satisfaction  with  the  level  of  communication  between  authors  end 
commenters  and  because  the  procedural  aspects  of  SADT  had  been  altered 
to  meet  the  needs  of  this  particular  application  of  the  technique,  the 

issue  of  process  data  and  implementation  was  not  pursued  in  subsequent 
evaluative  phases.  It  appears  that  the  traditional  procedural  mechanisms 
of  SADT  need  not  be  strictly  adhered  to  if  appropriate  alterations  can  be 
made  to  facilitate  participation  in  the  process  by  personnel  in  the  client 
organization.  The  fact  that  positive  project  benefits  were  derived  de¬ 
spite  less  than  strict  adherence  to  traditional  procedures  supports  this 
contention.  . 

Perceived  Effectiveness  and  Usefulness  of  SADT 

Examination  cf  the  results  of  the  three  evaluation  phases 
indicated  that  a  favorable  change  in  attitudes  towards  SADT  by  TRADOC 
personnel  had  occurred.  The  Phase  1  evaluation  concluded  that  the 
effectiveness  and  usefulness  of  SADT  had  not  been  clearly  demonstrated 
at  the  conclusion  of  Task  2  and  that  the  respondents  indicated  they 
believed  the  technique  was  a  good  one,  generally,  but  had  reservations 
about  the  ultimate  impact  of  its  application  at  TRADOC. 

The  Phase  2  evaluation,  which  followed  Task  3,  concluded  that 
attitudes  towards  the  SADT  project  and  its  results  were  positive  and  had 
changed  considerably  from  what  was  found  during  Phase  1.  The  most  con¬ 
sistently  cited  reason  for  the  more  positive  sentiments  found  in  the 
Phase  2  evaluation  was  that  the  Task  3  model  had  provided  an  Integrated 
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picture  of  the  total  Army  training  and  evaluation  system,  particularly 
the  interrelationships  between  evaluation,  system  development,  and 
training.  This  integration  had  not  been  as  vividly  portrayed  previously 
and  generated  considerable  enthusiasm  for  beginning  the  actual  work  of 
implementing  the  new  training  and  evaluation  system. 

Phase  3  evaluation  results  were  similar  to  those  found  in 
Phase  2.  Attitudes  towards  the  SADT  project  remained  positive.  Despite 
the  obstacles  encountered  in  beginning  the  actual  implementation  of 
the  new  training  and  evaluation  system,  the  Task  3  model  was  still  con¬ 
sidered  valuable  documentation  of  TRADOC 'a  best  thinking  on  how  the 
Army's  new  training  and  evaluation  system  should  operate.  Most  respon¬ 
dents  were  of  the  opinion  that  the  project  has  been  a  worthwhile  experience 
for  TRADOC  and  expressed  that  the  groundwork  had  been  laid  for  beginning 
actual  implementation  of  the  new  system. 

Thus,  TRADOC  personnel  were  originally  unsure  or  even  skeptical 
about  the  value  of  SADT  but  attitudes  towards  the  technique  shifted  as  the 
project  progressed.  Only  two  TRADOC  officers  questioned  during  the 
Phase  3  evaluation  stated  that  they  were  unaware  of  such  shifts  in  attitude 
across  the  project,  while  another  indicated  that  the  opinions  of  ScfTech 
personnel  had  changed  during  the  project  but  the  same  was  not  true  for 
TRADOC  personnel. 

To  validate  whether  shifts  in  attitude  toward  SADT  had  occurred 
at  TRADOC,  two  strategies,  one  quantitative  and  one  qualitative,  were 
employed.  Qualitative  data  were  gathered  (interview  protocols  and 
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responses  to  discussion  questions  from  questionnaires)  to  determine  if 
the  subjective  responses  of  specific  individuals  changed  across  evaluation 
phases. 


A  colonel  having  considerable  management  responsibility  for 
the  project  believed  that  at  the  end  of  Task  2  the  scope  of  the  project 
was  "far  too  global  to  be  of  any  practical  value"  and  favored  redirecting 
the  Task  3  effort  to  concentrate  on  particularly  "thorny"  here-and-now 
problems. 

His  attitude  was  expressed  in  the  following  comment  made  prior 

to  completion  of  the  Task  3  effort: 

1  can't  afford  to  spend  a  lot  of  my  time  working 
on  these  broad  general  goals.  I  have  my  own 
problems.  If  they  will  work  towards  my  particular 
problems  at  this  time,  fine,  but  I  can't  spend 
my  time  and  the  time  of  my  men  if  it's  not  going 
to  solve  my  problems. 

However,  following  dissemination  of  the  Task  3  model  the 
officer's  opinion  was  noted  to  havechanged  considerably  as  exemplified  by 
this  later  consent : 

For  some  time  we  thought  we  understood  different 
aspects  of  the  system,  but  for  the  first  time 
we  have  a  clear  picture  of  the  interrelationships 
of  different  functions  and  organisations  of  the 
Army  as  they  relate  to  training.  The  model 
systematised  and  coalesced  our  knowledge. 

When  questioned  during  the  Phase  3  evaluation  about  ways  the 
TRADOC  organization  had  benefitted  from  the  project,  this  individual 
stressed  that  the  work  completed  had  provided  TRADOC  with  "a  better  frame¬ 
work  within  which  to  accomplish  its  mission." 

Another  colonel  made  these  consents  following  the  conclusion  of 
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Task  2: 


We  went  Into  this  project  to  get  soneone  to  provide 
a  logical  layout  of  the  training  system  so  we  could 
get  a  handle  on  it.  (Individuals)  like  myself  haven't 
gotten  that  out  of  it  yet.  (Individuals)  like  myself  haven't 
work  will  be  of  any  use  at  this  time.... I 'll  wait  be¬ 
fore  saying  yes  or  no. 

Six  months  later  and  after  dissemination  of  the  Task  3  report, 

t’lis  same  individual  made  this  comment: 

Nothing  new  appeared  in  the  models  but  they  caused 
me  to  think  of  them  (training  issues)  in  a  different 
light.  The  further  delineation  of  what  evaluation 
is  has  changed  our  concept  of  evaluation  and  how  it 
interrelates  with  the  training  system. .. .This  year 
has  seen  the  realization  of  what  we've  been  (talking) 
about  for  three  years. 

The  above  comments  are  typical  of  the  way  people  described  the 
SADT  project  before  and  following  dissemination  of  the  Task  3  model. 

The  second  strategy  employed  to  determine  if  attitudes  had 
changed  during  the  course  of  the  project  was  to  compare  the  responses 
to  specific  quantitative  questions  from  the  Phase  1  and  Phase  3  question¬ 
naires.  Examination  of  these  instruments  (see  Appendices  B  and  D)  indi¬ 
cated  that  10  questions  were  common  to  both  Instruments.  Figure  7  pre¬ 
sents  the  intent  of  these  questions  and  the  mean  response  to  each  item 
across  evaluation  Phases  1  and  3. 

Figure  7  indicates  that  the  mean  responses  to  items  common  to 
the  Phase  1  and  Phase  3  questionnaires  were,  with  one  exception,  higher 
for  Phase  3  respondents  than  for  Phase  1  respondents.  Six  of  the  respon¬ 
dents  to  the  Phase  3  questionnaire  (n  *  12)  were  also  respondents  during 
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Mean  Responses  for  Selected  Items  of  Phase  1  and  Phase  3  Evaluation  Questionnaire 


Phase  1  (n  ■  13).  Relatively  large  shifts  (of  approximately  one  unit 
magnitude)  occurred  for  items  inquiring  whether  SADT  had  saved  time, 
whether  SADT  was  an  effective  design  and  analysis  method  and  whether 
the  progress  achieved  had  been  worth  resources  devoted  to  the  project. 
Moderate  shifts  (of  approximately  .5)  occurred  for  items  inquiring  about 
SADT' 8  facility  for  pvomoting  clearer  understanding  and  focusing  atten¬ 
tion.  These  results  further  serve  to  illuminate  the  nature  of  changes 
in  attitude  before  and  after  production  of  the  Task  3  model. 

To  further  specify  why  attitudes  towards  SADT  became  more 

positive  as  the  project  progressed,  TRADOC  officers  were  questioned  in 

the  Phase  3  evaluation  questionnlare  (Appendix  D,  Item  22)  as  to  why 

their  impressions  of  the  SADT  project  had  changed.  The  general  response 

given  was  that  at  the  conclusion  of  Task  2,  only  the  necessary  preliminary 

activities  had  been  completed  while  production  of  the  Task  3  model  was 

the  goal  of  the  Army's  participation  in  this  project.  The  following 

responses  are  representative: 

More  people  were  able  to  Identify  with  and  use  the 
general  model  described  in  the  Task  3  report. 

Since  Task  2  dealt  with  a  specific  weapons  system 
(the  tank),  many  officers  were  either  unfamiliar 
or  uninterested  in  the  Task  2  report.  The  general 
subject  of  the  training  system  is  more  widely 
understood  and  generated  a  greater  degree  of 
interest.  Those  who  understood  the  entire  project 
knew  that  the  meat  of  the  contract  was  in  Task  3 
end  that  Tasks  1  and  2  were  merely  preliminaries. 

The  Task  1  and  2  reports  were  not  promulgated  as 
solutions.  They  were  a  means  to  an  end.  The  Task  3 
report,  on  the  other  hand,  was  Intended  for  wider 
distribution.  Thus  improvement  in  opinion  should 
occur . 
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TRADOC  personnel  were  unsure  of  the  value  of  the  SADT  project 
during  the  preliminary  tasks  of  the  project.  The  Task  3  work  Integrated 


those  previous  efforts  and  produced  a  model  of  the  training  and  evaluation 
system  as  it  should  be,  which  was  the  TRADOC  goal  for  the  project.  The 
complexity  of  the  problem  created  initial  confusion  among  TRADOC  personnel 
as  to  the  goals  and  objectives  of  the  project,  and  because  the  primary 
project  goal  was  articulation  of  the  new  training  system,  it  was  not 
until  the  production  of  the  Task  3  model  that  individuals  could  see  that 
this  goal  was  attainable. 

This  conclusion  should  not  imply  that  evaluation  efforts  at  the 
conclusion  of  Task  2  came  too  early.  Rather,  the  Phase  1  evaluation  results 
were  critical  data  for  understanding  the  initial  ambiguity  und  skepticism 
inherent  in  projects  such  as  this  one  which  feature  the  novel  application 
of  a  specific  technique  to  a  highly  complex  system  problem. 

Factors  Influencing  Interpretation  of  Project-wide  Results 

The  following  discussion  presents  several  aspects  of  the  TRADOC 
organization  which  should  be  considered  in  drawing  conclusions  from  this 
study.  It  is  believed  that  these  factors  have  had  sufficient  impact  on 
these  results  to  warrant  discussion. 

A.  Lack  of  command  emphasis.  Throughout  the  project  TRADOC 
officers  have  stated  that  the  value  and  usefulness  of  the  SADT  project 
was  dependent  on  command  emphasis  and  that  the  project  has  had  leos  than 
full  support  from  the  upper  level  management  of  TRADOC.  This  lack  of 
comtand  emphasis  may  have  affected  some  individual's  opinions  of  and 
participation  in  the  project,  thus  hindering  the  accomplishment  of  project 
goals.  Although  not  measurable,  the  subtle  influence  this  variable  may 

have  had  on  the  conclusions  drawn  can  not  be  discounted. 
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B.  Acknowledgment  of  contribution.  This  factor  concern*  a  ten¬ 


dency  on  the  part  of  some  TRADOC  officers  to  diacount  the  contributions 
of  outside  contractors.  Although  not  measured  in  this  study,  the  impression 
of  the  evaluators  is  that  some  officers  would  have  little  good  to  say  about 
the  work  of  any  outside  contractor,  regardless  of  performance.  This 
relates  to  what  one  TRADOC  officer  called  "the  not  invented  here  syndrome." 
This  refers  to  the  belief  that  officers  will  tend  to  emphasize  the  work 
done  in  their  own  shop  and  dismiss  the  work  done  by  others. 

C.  Influences  on  the  status  of  training.  It  was  concluded  at 
the  end  of  the  Phase  2  evaluation  that  a  potentially  large  number  of  in¬ 
fluences  may  have  Interacted  to  produce  current  attitudes  at  TRADOC. 

The  SADT  project  represents  only  one  of  these  potential  influences.  The 
most  direct  response  from  an  officer  bearing  on  this  issue  was  from  a 
colonel  who  offered  the  following  comment: 

The  discussions  with  SofTech  and  examination  of 
diagrams  was  one  of  several  Intellectual  activities 
that  defined  and  described  where  we  were  heading  in 
TRADOC. 

It  is  not  possible  to  "tease  out"  the  amount  of  influence  the 
SofTech  project  has  had  in  bringing  about  the  Army's  current  conceptuali¬ 
zation  of  training  from  the  contribution  of  other  potential  influences. 

Thus,  the  data  gathered  for  this  evaluation  must  necessarily  be  inter¬ 
preted  in  light  of  the  fact  that  this  project  was  one  of  several  activities 
engaged  In  by  TRADOC  to  achieve  a  better  understanding  of  their  training 
system. 
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Su— ary  and  Conclusion; 


la  summary,  the  articulation  of  tha  Army  training  and  evaluation 
system  as  it  should  be  represents  the  most  tangible  direct  benefit  gained 
by  TRADOC  in  this  project.  The  Task  3  model  integrated  the  previously 
independently  conceptualised  areas  of  training,  system  developments,  and 
evaluation  in  an  understandable  and  practical  manner.  Nov  that  documen¬ 
tation  of  the  system  structure  has  been  accomplished  the  actual  work  of 
Implementing  the  new  conceptualisation  of  training  can  begin. 

At  a  minimum,  the  Task  3  model  has  provided  a  way  of  communi¬ 
cating  to  agencies  and  individuals  both  within  and  outside  TRADOC  the 
future  directions  planned  for  Army  training  and  evaluation.  Prior  to 
the  production  of  the  Task  3  model  the  training  developments  component 
of  TRADOC  had  not  been  able  to  fully  articulate  its  goals  and  intents. 

The  Task  3  model  provided  this  much  needed  documentation. 

Perhaps  the  most  disappointing  result  of  this  project  was  that 
only  a  limited  amount  of  technology  transfer  occurred  between  TRADOC 
and  SofTech.  Given  that  the  level  of  Army  participation  in  the  project 
was  less  than  originally  anticipated,  the  fact  that  the  project  has  had 
minimal  impact  on  TRADOC' s  general  analysis  and  planning  capabilities 
is  not  surprising.  Nora  specifically,  the  failure  of  TRADOC  to  provide 
additional  personnel  as  SADT  authors  limited  the  amount  of  expertise 
SofTech  could  pass  on  to  the  client  organisation. 

In  spite  of  the  decreased  level  of  participation,  a  small 
degree  of  technology  transfer  did  occur.  One  TRADOC  officer  (a  captain) 
trained  in  authoring  SADT  diagrams,  became  proficient  in  this  skill. 
Additionally,  two  colonels  having  management  responsibility  in  the 


training  development  branch  of  TRADOC  warn  expoaad  to  tha  ganaral  SADT 
approach  to  complex  system  problems.  From  discuaalcns  with  thaaa  indi¬ 
viduals  during  the  project,  it  is  beliavad  that  they  now  hava  a  battar 
perspective  from  which  to  manage  complex  analysis  and  design  problems. 

The  original  project  goals  were  that  SADT  would  be  used  in 
conjunction  with  TRADOC  to: 

(1)  identify  changes  in  Army  training  that 
would  significantly  increase  combat 
effectiveness; 

(2)  describe  how  Army  training,  testing,  and 
evaluation  programs  will  operate  after 
the  proposed  changes  are  accomplished;  and 

(3)  plan  how  the  changes  in  Army  training 
will  be  undertaken  and  how  progress  will 
be  monitored  and  evaluated. 

The  conclusion  drawn  from  all  evaluation  data  collected  from  the  study 
is  that  goals  1  and  2  have  been  accomplished  to  the  satisfaction  of 
relevant  TRADOC  personnel. 

Goal  3,  however,  was  only  minimally  realized,  although  this 
was  not  due  to  shortcomings  in  the  SADT  technique  or  In  the  efforts  of 
SofTech  or  TRADOC.  The  actual  implementation  plan  was  deferred  by  mutual 
consent  and  replaced  by  a  more  pragmatic  intermediate  step.  It  was  not 
until  the  Task  3  model  had  been  produced  that  the  task  of  Implementing 
the  new  system  could  be  fully  appreciated.  It  was  then  decided  that  Task  4 
could  better  be  spent  attacking  more  specific  problems  that  stood  as 
obstacles  to  subsequent  implementation  of  the  full  system.  Given  this 
redirection,  the  failure  to  develop  a  plan  for  implementing  changes  and 
monitoring  progress  can  not  be  seen  as  a  limitation  of  SADT  but  rather 
as  a  reflection  of  the  complexity  of  the  context  in  which  the  project  occurred. 
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Thus,  It  is  concluded  that  further  applications  of  SADT  such  as 
the  one  described  in  this  report  would  be  worthwhile.  The  basis  for  this 
conclusion  is  not  that  there  is  any  "magic"  in  the  SADT  diagrams  or  pro¬ 
cedures  but  rather  that  they  provide  a  mechanism  for  documenting  the  best 
thinking  of  bright  Individuals  in  the  client  organisation  and  interrelating 
important  concepts  in  a  clear  and  practical  manner. 

It  does  not  appear  that  the  project  created  any  new  knowledge 
about  specific  aspects  of  the  training  system  but  rather  that  it  integrated 
and  communicated  what  was  known  about  training  in  a  coherent  and  useful 
way.  This  conclusion  is  best  summarised  by  an  officer  when  he  said, 

"(The  project)  forced  TRADOC  to  explain  the  system  used  to  outside  agents. 
This  is  a  remarkably  useful  exercise  for  smug  persons  who  think  they 
understand  what  they  do." 
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Appendix  A 

Evaluative  Dimensions  Derived  from  the  SofTech  Proposal 

I.  Communicative  Qualities 

While  explicit  claims  were  made  about  the  communicative  qualities  of  the 
technique,  almost  all  claims  at  least  Implied  communication.  Communication  ap¬ 
peared  to  be  one  of  the  underlying  goals  of  the  SADT  process.  There  are  three 
subdimensions  relevant  to  evaluation  of  the  communicative  qualities  of  SADT: 

(a)  Does  the  author  communicate  effectively  with  the  reader  through  SADT  dia¬ 
grams?;  (b)  Is  communication  enhanced  among  relevant  personnel  in  the  project 
by  following  the  SADT  discipline?;  and  (c)  Does  the  usefulness  of  the  technique 
derive  largely  from  the  technique  itself,  l.e.  from  blueprint-like  drawings,  or 
from  the  communication  between  individuals  engendered  by  the  process  of  using 
the  technique? 

A.  Communicative  Quality  of  the  Diagrams.  Are  SADT  diagrams  readable  by 
Army  personnel  after  training  provided  by  SofTech?  Is  the  message  of  a  diagram 
clear  and  unambiguous  to  the  reader?  Disregarding  the  accuracy  of  the  diagram, 
does  the  Army  reader  feel  that  he  understands  what  the  author  is  trying  to  say? 
Are  the  diagrams  more  communicative  than  standard  verbal  text?  Do  TRADOC  per¬ 
sonnel  believe  they  would  have  understood  the  author's  message  as  easily 
through  written  text  or  dialog? 

B.  Communicative  Quality  of  the  SADT  Discipline.  Has  the  SADT  discipline 
(the  iterative  author-coonenter  cycle)  facilitated  conunication  between  the 
diverse  personnel  participating  in  the  project?  Is  the  comnenter  cycle  more 
effective  than  written  communication  or  conversation?  Has  the  SADT  discipline 
facilitated  communication  between  Army  personnel  about  the  problems  of  rapidly 
changing  training  procedures?  Has  the  discipline  helped  focus  attention  on 
the  relevant  factors  so  that  meaningful  dialog  could  occur  within  TRADOC?  To 
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what  uttnt  haa  tha  application  of  SADT  at  TRAOOC  lad  to  a  common  conceptuallsa- 
tlon  of  tha  problaaT  Do  Any  paraonnal  ballava  thay  are  Bora  abla  to  coamuni- 
cate  among  themselves  about  thalr  tasks,  goals,  and  problems  aa  a  raault  of  the 
application  of  SADT? 

C.  Impact  of  lap  roved  Communication.  If  coamunicatlon  about  tha  problems 
at  hand  haa  iaproved  at  TRADOC  (A  and  B  above)*  haa  thla  improved  communication 
helped  focus  attention  on  critical  variables  impacting  the  effectiveness  of 
Aray  training?  Has  thla  improved  communication  led  to  a  keener  understanding 
of  the  problems  facing  the  training  coamand  .  .  .  to  a  grasp  of  possible  solu¬ 
tions  to  those  problems?  To  vhat  extent  are  the  benefits  of  the  SADI  project 
attributable  to  the  Impact  of  SADT  on  coamuni  cat  ions  within  TRADOC? 

II.  Quality 

The  proposal  made  several  claims  to  the  effect  that  SADT  will  Improve 
the  quality  of  the  work  at  TRADOC.  While  these  quality  claims  were  sometimes  too 
vague  to  measure,  several  claims  did  cluster  into  three  groups  that  are  logically 
related  to  quality:  accuracy,  quality  control,  and  efficiency  (time  savings). 
These  three  subdimenalons  form  the  Quality  dimension. 

A.  Accuracy.  Do  Army  personnel  consider  the  SADT  models  to  be  accurate 
representations  of  the  content  modeled?  Is  the  overview  model  developed  in 
Task  1  a  credible  representation  of  the  new  training  progress  to  those  who  are 
responsible  for  the  conceptualisation  and  dealgn  of  these  new  programs?  Similar¬ 
ly,  is  the  model  of  the  tank  weapon  system  developed  in  Task  2  sufficiently 
isomorphic  with  that  system  for  Army  users  to  develop  confidence  in  the  model 
and  to  use  it  to  deepen  their  understanding  of  the  weapon  system? 

B.  Quality  Control.  Built  into  the  SADT  discipline  is  an  iterative  re¬ 
view  process  that  supposedly  ensures  the  quality  (accuracy,  consistency,  and 
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completeness)  of  the  ongoing  work.  Do  Army  personnel  perceive  this  method  of 
quality  control  as  effective?  Do  they  believe  that  this  process  actually  ensures 
the  quality  of  the  final  draft  of  a  diagram?  If  they  do  not  see  the  process 
as  effective,  is  it  because  of  the  author's  Insensitivity  to  cowents  by  com- 
menters  .  .  .  lack  of  interest  and/or  input  by  cosnenters  .  .  .  Inconsistent 
feedback  to  author  .  .  .  poor  information  grasp  by  experts  of  the  content 
modeled  .  .  .  and/or  limitations  or.  the  way  feedback  Is  given? 

C.  Efficiency.  Several  claims  were  ir'de  about  the  reduction  in  time 
resulting  from  application  of  SADT  in  completing  a  structured  analysis  and  design 
problem.  Do  the  Army's  perceptions  support  these  claims?  Does  the  Iterative 
review  cycle  of  the  SADT  discipline  save  time?  Is  it  a  more  efficient  way  (in 
terms  of  time)  to  communicate  ideas?  Does  the  discipline  seem  more  time  ef¬ 
ficient  early  In  the  modeling  (A-0  and  A#)  or  later  in  the  modeling,  i.e. ,  at 
more  detailed  levels? 

At  the  end  of  Task  2,  do  Army  personnel  consider  the  amount  of  prog¬ 
ress  made  on  the  problem  as  a  result  of  SofTech's  input  worth  the  time  devoted 
to  the  project?  Do  they  consider  this  amount  of  time  a  savings  or  deficit  rela¬ 
tive  to  the  time  it  would  have  taken  to  reach  the  same  level  of  output  by  more 
traditional  methods? 

III.  Usefulness 

Thl3  dimension,  reflecting  the  largest  single  cluster,  represents  claims 
about  the  utility  or  usefulness  of  the  technique.  Taken  together,  these  claims 
explicitly  state  that  SADT  is  more  than  a  decomposition  technology  or  engineer¬ 
ing  drawing  system,  i.e.,  that  it  will  facilitate  the  planning,  implaaNmtatlon, 
management,  and  evaluation  capabilities  of  TRADOC  personnel  by  providing  con¬ 
ceptual  insights  into  the  problems  being  modeled. 
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The  evaluation  of  this  dimension  will  be  conducted  at  two  different  stages: 
(a)  an  evaluation  of  SADT's  usefulness  in  Tasks  1  and  2  (incorporated  in  this 
evaluation  report),  and  (b)  &ien  the  Army  personnel's  experience  with  SADT  in 
Tasks  1  and  2,  their  perception  of  the  usefulness  of  SADT  when  applied  in 
Tasks  3  and  4  (not  scheduled  for  completion  prior  to  the  preparation  of  this 
report) . 

A.  Usefulness  in  Tasks  1  and  2.  Did  the  models  produced  in  Task  1  lead 
to  a  clearer  understanding  of  the  problems  being  modeled?  Will  these  diagrams 
be  helpful  in  planning  changes  in  Army  training?  Did  the  model  define  what 
types  of  changes  are  being  planned? 

In  T/^sk  2,  did  the  training  innovations  having  the  greatest  Impact  on 
combat  readiness  and  battlefield  effectiveness  become  sufficiently  visible  as 
claimed?  By  the  end  of  Task  2,  did  SADT  contribute  to  the  isolation  of  inno¬ 
vations  which  are  critical  to  Army  training? 

B.  Usefulness  in  Subsequent  Tasks.  Do  Army  personnel  believe  SADT  will 

be  effective  in  elaborating  the  Task  1  model?  Given  their  experience  with  SADT 
in  Tasks  1  and  2,  do  they  believe  the  more  thorough  model  produced  in  Task  3 
and  the  innovation  plan  (Task  4^  will  Increase  TRADOC's  analysis,  planning,  and 
management  capabilities?  In  addition  to  their  utility  for  the  present  evalua¬ 
tion  effort  questions  such  as  these  ultimately  will  be  used  to  determine 
whether  the  consensus  of  opinion  at  TRADOC  supports  the  continuation  of  the 
project  and/or  the  use  ot  SADT  in  expanded  contexts. 

IV.  Specificity 

This  dimension  reflects  the  Army's  perception  of  SADT's  ability  to  produce 
models  that  meet  a  specific  need,  such  as  describing  what  functions  a  system 
must  perform,  specifying  how  a  system  should  be  designed,  or  how  a  system  should 
be  managed  or  maintained.  Answers  to  these  types  of  questions  provided  some 
initial  information  bearing  on  a  decision  about  expanded  usage  of  SADT  in 
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the  DoD  environment. 


A  second  ares  of  Specificity  is  whether  Any  personnel  can  obtain  specific 
information  from  the  models.  If  an  Army  officer  needs  to  know  something 
specific  about  a  system,  can  he  look  at  the  SADT  model  of  that  system  and  find 
what  he  needs?  Could,  for  example,  a  microfiche  SADT  model  file  serve  as  a 
valuable  reference  for  Army  training  personnel? 

These  evaluative  dimensions  and  their  respective  claims  are  summarised 
in  the  following  table. 
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Appendix  B 


Phase  1  Evaluation  Questionnaire 

Current  Position 

Months  in  Current  Position 

Total  length  of  experience  with  Any  training 

and  testing  programs _ 

Have  you  taken  the  So f Tech  Reader  Course?  yes  f~T  no  /  / 

Have  you  taken  the  SofTech  Comenter  Course?  yes  /  /  no  /  / 

1.  To  what  extent  do  you  feel  you  were  adequately  trained  to  read 
SADT  diagraes? 

1.  _ not  trained  at  all 

2.  _ Doorly  trained 

3.  _ trained  to  a  working  knowledge 

4.  _ fairly  well  trained 

5.  _ expertly  trained 
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2.  To  what  extant  do  you  feel  yju  were  adequately  trained  to  coien 
on  SADT  diagrams? 

1.  _ not  trained  at  all 

2.  _ poorly  trained 

3.  _ trained  to  a  working  knowledge 

4.  _ fairly  well  trained 

5.  _ expertly  trained 


3.  In  general,  to  what  extent  were  the  SADT  dlagraae  clear  and 
unambiguous?  (Complete  each  column  aeparately.) 

first  revised  final 

drafts  versions  versions 


1.  Diagrams  were  always 
ambiguous 

2.  Diagrams  were  mostly 
ambiguous 

3.  Diagrams  were  occasion¬ 
ally  clear 

4.  Diagrams  were  nastly  clear 

5.  Diagrams  were  always  clear 


4.  To  what  extent  do  you  think  it  is  necessary  that  written  text 
containing  a  more  complete  explanation  of  the  content  modeled 
accompany  SADT  diagrams  contained  in  reader  kits? 

1.  _ not  necessary 

2.  _ might  be  slightly  helpful 

3.  _ probably  of  some  help 

4.  _ would  be  fairly  helpful 

5.  _ would  be  very  helpful 
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S.  In  your  opinion,  do  SADT  raader  kit*  allow  idequata  communication 
bacwaan  author  and  common tar  or  ahould  discussion  batwaan  author 
and  raadar  ba  a  raquirad  part  of  tha  SADT  process? 

1.  _ SADT  diagram  are  an  adaquata  mans  of  communication; 

no  discussion  is  necessary 

2.  _ discussion  between  author  and  raadar  at  tha  request  of 

either  party  (the  present  procedure)  is  adaquata 

3.  _ inclusion  of  required  discussion  between  author  and 

reader  would  be  helpful 

4.  _ Inclusion  of  required  discussion  between  author  and 

reader  is  essential 


6.  To  what  degree  do  you  believe  SADT  diagram  are  an  effective  way 
of  communicating  to  trained  readers  the  substance  of  providing 
Amy  training,  testing,  and  evaluation  program? 

1.  _ not  effective 

2.  _ rarely  effective 

3.  _ 8  one  times  effective 

4.  _ fairly  effective 

5.  _ very  effective 


7.  To  your  knowledge  »  to  what  extent  have  SADT  diagram  generated 
either  forml  or  Informal  connunlcatlon  about  Army  training 
procedures  among  TRADOC  personnel  directly  involved  with  the  SADT 
project? 

1.  _ none  at  all 

2.  _ a  little 

3.  _ some 

4.  _ fair  amount 

5.  _ very  significant  amount 
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To  your  knowledge,  to  whet  extent  have  SADT  diagrams  generated 
either  formal  or  Informal  coamunlcatlon  about  Army  training 
procedir  ea  aaong  TRADOC  personnel  not  directly  Involved  with  the 
SADT  project? 

1.  _ none  at  all 

2.  _ a  little 

3.  _ some 

4.  _ fair  amount 

5.  _ very  significant  amount 


To  what  extent  have  SADT  diagrams  helped  focus  your  attention  on 
training  and  testing  variables  which  you  had  not  considered 
previously? 

1.  _ not  at  all 

2.  _ very  little 

3.  _ somewhat 

4.  _ to  a  fair  extent 

5.  _ very  much 


To  what  extent  do  you  believe  tne  benefits  of  the  SADT  project  at 
TRADOC,  if  any,  are  due  to  the  diagrams  or  models  theamelves,  i.e., 
the  tangible  products  or  drawings,  as  opposed  to  the  process  of 
communication  between  TRADOC  staff  members  which  may  have  been 
stimulated  by  the  modeling  process?  If  you  feel  no  benefits 
accrued  at  all  relevant  to  this  question,  check  only  the  following: 

/  /  no  benefits  at  all 

1.  _ benefits  entirely  product-diagram  related 

2.  _ benefits  mostly  product-diagram  related 

3.  _ benefits  split  about  evenly  between  products  produced 

and  processes  stimulated 

4.  _ benefits  mostly  process  related 

5.  _ benefits  entirely  process  related 
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11. 


To  what  extant  do  tha  revised  dlagraas  that  you  have  seen  of  the 
new  Amy  training  and  tasting  program  accurately  represent  the 
content  rndeled? 

/  /  not  enough  information  to  judge 

1.  not  accurate 

2.  _ rough  approximations 

3.  _ variable  in  accuracy,  depending  on  content  modeled 

4.  _ fairly  accurate 

5.  _ extremely  accurate 


12.  To  what  extent  do  the  revised  diagram  that  you  have  seen  of  the 
tank  weapon  ays ten  accurately  represent  that  syatec? 

/  /  not  enough  information  to  judge 

1.  _ not  accurate 

2.  _ rough  approximations 

3.  _ variable  in  accuracy,  depending  on  content  modeled 

4.  _ fairly  accurate 

5.  _ extremely  accurate 

13.  Is  the  author-conmenter  review  cycle  used  by  SofTech  in  the 
production  of  SADT  diagrams  an  effective  means  of  ensuring  the 
quality  of  SADT  models? 

1.  _ not  at  all  effective 

2.  not  very  effective 

3.  _ somewhat  effective 

4.  _ fairly  effective 

5.  _ very  effective 
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14.  How  confident  are  you  that  this  method  of  quality  control  guarantees 
that  the  final  diagram  will  be  as  complete  and  accurate  as  possible? 

1.  _ not  at  all  confident 

2.  _ very  little  confidence 

3.  _ some  confidence 

4.  _ fairly  confident 

5.  very  confident 

15.  What  degree  of  confidence  do  you  have  that  your  written  comments 
on  SADT  diagrams  were  actually  taken  into  account  in  producing 
final  draft  models? 

1.  _ no  confidence  at  all 

2.  _ low  confidence 

3.  _ some  confidence 

4.  _ fairly  high  degree  of  confidence 

5.  _ very  high  degree  of  confidence 

16.  In  your  opinion,  to  what  extent  has  communicating  to  SofTech 
authors  by  way  of  the  author-commenter  cycle,  rather  than  by 
discussion  and  written  text,  resulted  in  a  time  savings? 

1.  _ no  time  was  saved 

2.  _ a  little  time  was  saved 

3.  _ some  time  was  saved 

4.  _ a  fair  amount  of  time  T.as  saved 

5.  _ a  great  deal  of  time  was  saved 
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17.  To  what  extent  has  the  application  of  SADT  at  TRADOC  saved  time 
relative  to  other  approaches  (e.g. ,  ln-house  TRADOC  committees, 
outside  consultants)  that  might  have  been  employed  to  communicate 
Army  training  procedures? 


l._ 

_ has 

not  saved  time 

2 . _ 

_ has 

saved 

very  little 

time 

3 . _ 

_ has 

saved 

some  time 

4 . _ 

_ has 

saved 

a  fair  amount  of  time 

5. 

has 

saved 

considerable 

time 

18.  How  cost-efficient  do  you  believe  SADT  is  compared  to  other 
techniques  having  the  same  general  purpose  with  which  you  are 
familiar? 

/  /  unfamiliar  with  other  techniques 

1.  _ all  others  are  superior  to  SADT 

2.  _ many  others  are  superior  to  SADT 

3.  _ others  and  SADT  are  about  the  same 

4.  _ a  few  others  are  superior  to  SADT 

5.  _ SADT  is  superior  to  all  others 

19.  In  your  opinion,  approximately  what  percentage  of  actual  time  could 

be  saved,  if  any,  by  using  SADT  on  a  design  and  analysis  task 
instead  of  traditional  procedures?  _ X 
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20.  To  what  extent  did  the  Task  1  model  of  Army  training  provide  you 
with  a  clearer  understanding  of  the  Army's  training,  testing,  and 
evaluation  programs? 

/  /  unfamiliar  with  Task  1  model 

1.  _ less  understanding  than  before  project 

2.  _ same  understanding  as  before  project 

3.  _ a  slightly  better  understanding 

4.  _ a  noticeably  better  understanding 

5.  _ a  much  better  understanding  ■ 


21. 


Do  you  believe  the  SADT  models  produced  by  SofTech  thus  far  will 
actually  be  used  in  planning  needed  changes  in  Army  training? 

1.  will  not  be  used 


2 .  _ will 

3.  _ will 

4.  will 

5.  will 


be  used  to  a  small  extent 
be  used  9ome,  depending  on  content  modeled 
be  used  to  a  fair  extent 
be  used  extensively 


22.  To  what  extent  do  you  believe  the  SADT  diagrams  produced  thus  far 
will  actually  help  in  deriving  training  requirements? 


l.__ 

_ will 

not  help 

2. _ 

_ will 

help  to  a  small  extent 

3. _ 

_ will 

help  some 

4. _ 

_ will 

help  noticeably 

5 . _ 

_ will 

help  considerably 
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23. 


In  your  opinion,  can  SADT  be  effectively  used  to  elaborate  upon 
already  completed  diagraas  and  to  accurately  articulate  an  extended 
final  model  of  A  nay  training  and  testing  during  the  next  phase  of 
the  project  (Task  3)7 

1.  _ definitely  not 

2.  _ probably  not 

3.  uncertain 

4.  _ probably  yea 

5.  _ definitely  yes 


24.  To  what  extent  will  SADT  models  actually  increase  TRADOC's  analysis, 
planning  and  management  capabilities?  (Complete  each  column 
separately. ) 

Analysis  Planning  Management 

1.  none  _  _  _ 

2.  a  little  _ _ _ 

3 .  some  _ _ _ 

4.  a  fair  amount  _  _  _ 

5.  very  much  _  _  _ 

25.  Do  you  believe  that  SADT  can  help  identify  training  requirements 
that  have  a  significant  effect  on  combat  readiness  and  battlefield 
effectiveness? 

1.  _ definitely  not 

2.  _ a  little 

3 .  _ somewhat 

4.  _ to  a  fair  extent 

5.  _ to  a  significant  extent 


119 


26.  In  your  opinion,  would  a  complete  SADT  model  of  Army  training  serve 
as  a  valuable  future  reference  for  Army  personnel  responsible  for 
providing  training? 

1.  _ would  not  be  of  any  value 

2.  _ would  be  of  little  value 

3.  _ would  be  of  some  value 

A. _ would  be  fairly  valuable 

5. _ would  be  very  valuable 

27.  How  practical  do  you  believe  it  is  for  TRADOC  personnel  to  use  SaDT 
for  most  design  and  analysis  problems  facing  the  Training  and 
Doctrine  Command? 

1.  _ very  impractical 

2.  _ fairly  impractical 

3.  _ marginally  practical 

A. _ fairly  practical 

5. _ very  practical 

28.  How  confident  were  you  in  the  comments  you  made  on  SADT  diagrams? 

1.  _ not  at  all  confident 

2.  _ barely  confident 

3.  _ somewhat  confident 

A. _ fairly  confident 

5. _ very  confident 
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29. 


Of  the  diagrams  you  read,  approximately  what  percentage  were  in 
content  areas  in  which  you  felt  you  did  not  have  sufficient  experi¬ 
ence  to  make  meaningful  consents?  If  you  would  like  to  be  more 
precise  in  your  estimate,  do  so  in  the  space  provided:  _ X. 

1. _ 0-20% 

2.  _ 21-40% 

3.  _ 41-60% 

4.  _ 61-80% 

5.  81-100% 


30.  About  how  often  did  you  consult  other  TRAD0C  personnel  before 

consenting  on  an  SADT  diagram?  If  you  would  like  to  be  more 
precise  in  your  estimate,  do  so  in  the  space  provided:  _ X. 

1. _ 0-20% 

2.  _ 21-40% 

3.  _ 41-60% 

4.  _ 61-80% 

5.  _ 81-100% 

31.  At  this  time,  how  critical  do  you  believe  it  is  to  implement 
changes  in  Army  training? 

1.  _ no  need  presently  exists 

2.  _ needs  to  be  done  but  not  immediately  (low  priority) 

3.  _ somewhat  critical  (medium  priority) 

4.  _ fairly  critical  (high  priority) 

5.  _ very  critical  (urgent  priority) 
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32.  To  what  extent  do  you  believe  SADT  model*  will  provide  the  impetus 
for  changes  in  Army  training  and  testing  programs? 

1.  will  not  provide  any  impetus  for  change 

2.  _ will  contribute  to  change  to  a  small  extent 

3.  _ will  provide  some  impetus  for  change 

4.  will  contribute  to  change  to  a  fairly  large  extent 

5.  _ will  provide  the  major  impetus  for  change 

33.  To  what  extent  will  SADT  be  useful  in  identifying  existing 
organizational  inefficiencies  in  the  current  training  and 
testing  procedures? 

1.  _ no  use 

2.  _ not  much  use 

3.  _ some  use 

4.  _ a  fair  amount  of  use 

5.  _ a  great  use 

34.  In  your  opinion,  will  the  application  of  SADT  lead  to  conceptual 
insights  about  alternative  methods  of  conducting  Army  training 
and  testing  programs? 

1. _ not  at  all 

f  2. _ very  little 

3.  _ uome 

4.  _ to  a  fair  extent 

5.  _ to  a  great  extent 

! 

I 

* 

i 

\ 
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35.  In  your  opinion,  will  the  use  of  SADT  ultimately  effect  the 
training  concepts  currently  held  by  TRADOC? 

1.  _ not  at  all 

2.  _ very  little 

3.  _ to  some  extent 

4.  _ to  a  considerable  extent 

5.  _ to  a  great  extent 

36.  In  your  opinion,  would  it  be  useful  to  TRADOC* 8  mission  to 
train  some  Army  personnel  to  be  SADT  authors? 

1.  _ no  use 

2.  _ slightly  useful 

3.  _ of  some  use 

4.  _ fairly  useful 

5.  very  useful 

37.  In  your  opinion,  has  the  amount  of  progress  made  by  this  project 
on  the  problems  facing  TRADOC  been  worth  the  amount  of  time  Army 
personnel  have  devoted  to  it? 

1.  _ definitely  haa  not  been  worth  the  time 

2.  _ generally  has  not  been  worth  the  time 

3.  _ has  varied  with  the  content  being  modeled 

4.  _ generally  has  been  worth  the  time 

5.  _ all  time  spent  has  been  worthwhile 
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38.  To  what  extent  did  the  ‘Vaak  2  aodel  of  the  tank  weapon  system 
provide  you  with  a  clearer  underatanding  of  the  Army's  training, 
testing,  and  evaluation  programs? 

/  /  unfamiliar  with  Taak  2  aodel 

1.  _ leas  underatanding  than  before  project 

2.  _ same  underatanding  aa  before  project 

3.  _ a  slightly  better  underatanding 

4.  _ a  noticeably  better  understanding 

5.  _ a  much  better  understanding 


Please  respond  to  the  following  questions  by  writing  your  answers 
in  the  space  provided.  If  you  need  more  space,  continue  on  the  extra 
pages  at  the  end  of  this  booklet  or  attach  an  additional  sheet.  While 
some  of  the  questions  could  be  responded  to  with  a  brief  coanent,  I 
would  like  to  ask  that  you  elaborate  on  your  responses  whenever 
possible. 


39.  From  your  experience  in  revising  SADT  diagrams,  can  you  recall  if 
there  were  general  content  areas  in  which  diagrams  were  particu¬ 
larly  accurate  and  other  areas  where  they  were  particularly 
inaccurate?  What  variables,  in  your  opinion,  might  account  for 
differences  in  the  accuracy  of  SADT  diagrams? 


40.  Could  you  identify  any  specif ic  benefits  which  nay  have  accrued 
to  the  Amy  as  a  result  of  its  participation  in  this  project 
thus  far? 


n 

•*  * 

i. 

i. 


41.  Drawing  on  your  experience  with  SADT,  what  are  a  few  of  its  major 
strengths  and  weaknesses? 


i: 

i: 

i: 

r. 

r. 

r 

i: 

£ 
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43. 


To  your  knowledge,  have  there  been  any  obstacles  which  aay  have 
prevented  SofTech  fron  doing  the  best  job  possible  at  TRADOC? 
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Appendix  C 


Phase  2  Structured  Interview  Form 


1.  (7)*  To  your  knowledge,  to  what  extent  has  the  SofTech  project  fac¬ 

ilitated  either  formal  or  Informal  communication  about  Army 
training  procedures  among  TRADOC  personnel  directly  involved 
with  the  SADT  diagrams  (commenting  or  authoring)? 

1.  _ none  at  all 

2.  _ a  little 

3.  _ _  some 

4.  _  fair  amount 

5.  _  very  significant  amount 

2.  (8)  To  your  knowledge,  to  what  extent  has  the  SofTech  project  fac¬ 

ilitated  either  formal  or  informal  conmmication  about  Army 
training  procedures  among  TRADOC  personnel  not  directly  involved 
with  the  SADT  diagrams  (commenting  or  authoring)? 

1.  _ _  none  at  all 

2.  _ a  little 

3.  _ some 

4.  __  fair  amount 

5.  _  very  significant  amount 

3.  (9)  To  what  extent  has  the  SofTech  project  helped  focus  your  atten¬ 

tion  on  training  and  testing  variables  which  you  had  not  con¬ 
sidered  previously? 

1.  _  not  at  all 

2.  _  very  little 

3.  _ somewhat 

4.  _  to  a  fair  extent 

5.  _ very  much 


^Question  number  on  the  Phase  1  questionnaire 
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4.  (11)  To  what  extent  do  the  revised  diagrams  that  you  have  seen  of 

the  new  Army  training  and  testing  programs  accurately  represent 
what  the  Army  training  and  evaluation  system  should  be? 

/ _ /  not  enough  information  to  judge 

1.  _  not  accurate 

2.  _  rough  approximations 

3.  _  variable  in  accuracy,  depending  on  content  modeled 

4.  _  fairly  accurate 

5.  _ extremely  accurate 

5.  (20)  To  what  extent  will  the  Task  3  model  lead  to  a  clearer  under¬ 

standing  of  Army  training,  testing,  and  evaluation  programs? 

1.  _  less  understanding  than  before  project 

2.  __  same  understanding  as  before  project 

3.  _  a  slightly  better  understanding 

4.  _  a  noticeably  better  understanding 

5.  _  a  much  better  understanding 

6.  (21)  Do  you  believe  the  SADT  models  produced  by  SofTech  thus  far  will 

actuully  be  used  in  planning  needed  changes  in  Army  training? 

1.  _ will  not  be  used 

2.  _  will  be  used  to  a  small  extent 

3.  _  will  be  used  some,  depending  on  content  modeled 

4.  _  will  be  used  to  a  fair  extent 

5.  _  will  be  used  extensively 
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7.  (22)  To  what;  extent  do  you  believe  the  SADT  diagrams  produced  thus 
far  will  actually  help  to  establish  training  requirements? 


1.  _ will  not  help 

2.  _  will  help  to  a  snail  extent 

3.  _ will  help  some 

4.  _  will  help  noticeably 

5.  _ _  will  help  considerably 

8.  (23)  Have  the  Task  3  models  effectively  elaborated  those  developed 

in  Tasks  1  and  2? 

1.  _  definitely  not 

2.  _  probably  not 

3.  _  uncertain 

4.  _ _  probably  yes 

5'.  _  definitely 

9.  (24)  To  what  extent  will  SADT  models  actually  increase  TRADOC's 

analysis,  planning,  and  management  capabilities?  (Complete 
each  column  separately.) 

Planning  Analysis  Management 

1 .  none  _  _  _ 

2.  a  little  _  _  _ 

3 .  some  _  _  _ 

4.  a  fair  amount  _  _  _ 

5.  very  much  _  _____ 
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10.  (26)  In  your  opinion,  would  a  complete  SADT  model  of  A ray  training 

serve  as  a  valuable  future  reference  for  A ray  personnel  respon¬ 
sible  for  providing  training? 

1.  _  would  not  be  of  any  value 

2.  _  would  be  of  little  value 

3.  _ would  be  of  some  value 

4.  _  would  be  fairly  valuable 

5.  _  would  be  very  valuable 

11.  (27)  How  practical  do  you  believe  it  is  for  TRADOC  personnel  to  use 

SADT  for  most  design  and  analysis  problems  facing  the  Training 
and  Doctrine  Command? 


1.  _ _ very  impractical 

2.  _  fairly  impractical 

3.  _ marginally  practical 

4.  _  fairly  practical 

5.  _  very  practical 

12.  To  what  extent  to  you  believe  SADT  models  will  be  useful  in 

bringing  about  changes  in  Army  training  and  testing  programs? 
(Not  included  in  original  questionnaire.) 

1.  will  not  provide  any  impetus  for  change 

2.  _  will  contribute  to  change  to  a  small  extent 

3.  _  will  provide  some  impetus  for  change 

4.  _  will  contribute  to  change  to  a  fairly  large  extent 

5.  _  will  provide  the  major  impetus  for  change 
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13.  (33)  Has  SADT  been  useful  in  identifying  existing  organisational 

inefficiencies  in  the  current  training  and  testing  procedures? 

1.  _ no  use 

2.  _ not  much  use 

3.  _  some  use 

4.  _ a  fair  amount  of  use 

5.  _ a  great  use 

14.  (34)  Has  the  application  of  SADT  lead  to  conceptual  insights  about 

alternative  methods  of  conducting  Army  training  and  evaluation 
programs? 

1.  _  not  at  all 

2.  _  very  little 

3.  _  some 

4.  _  to  a  fair  extent 

5.  _  to  a  great  extent 

15.  (35)  Has  the  use  of  SADT  had  an  impact  on  the  training  concepts 

currently  held  by  TRADOC? 

1.  _  not  at  all 

2.  _ very  little 

3.  _  to  some  extent 

4.  _ _  to  a  considerable  extent 

5.  _  to  a  great  extent 
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16.  (36)  In  your  opinion,  would  it  be  useful  to  TRADOC's  mission  to 
train  bosk  Army  personnel  to  be  SADT  authors? 


1.  _ no  use 

2.  _  slightly  useful 

3.  _ of  some  use 

4.  _  fairly  useful 

5.  _ very  useful 

17.  (37)  In  your  opinion,  has  the  amount  of  progress  made  by  this  pro¬ 
ject  on  the  problems  facing  TRADOC  been  worth  the  resources 
the  Army  has  devoted  to  it? 

1.  _ definitely  has  not  been  worth  the  time 

2.  _  generally  has  not  been  worth  the  time 

3.  _  has  varied  with  the  content  being  modeled 

4.  _  generally  has  been  worth  the  time 

5.  ___  all  time  spent  has  been  worthwhile 
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Appendix  D 

Phase  3  Evaluation  Questionnaire 

Check  the  alternative  most  congruent  with  your  sent latent a  to  each  question. 
Feel  free  to  elaborate  on  your  responses  In  the  additional  apace  provided. 

1.  To  what  degree  do  you  believe  SADT  diagrams  are  an  effective  way 
of  communicating  to  trained  readers  the  substance  of  providing 
Army  training,  testing,  and  evaluation  programs? 


1. 

not  effective 

2. 

rarely  effective 

3. 

sometimes  effective 

4. 

fairly  effective 

5. 

very  effective 

2.  To  your  knowledge,  to  what  extent  did  the  SADT  project  generate 
either  formal  or  informal  communication  about  Army  training  pro¬ 
cedures  among  TRADOC  personnel? 

1. 

none  at  all 

2. 

a  little 

3. 

some 

4. 

fair  amount 

5. 

very  significant  amount 

3.  To  what  extent  did  SADT  diagrams  help  focus  attention  on  training 
and  testing  variables  which  had  not  been  considered  previously? 

1. 

not  at  all 

2. 

very  little 

3. 

somewhat 

4. 

to  a  fair  extent 

5. 

very  much 
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To  what  extent  did  SADT  models  elucidate  the  interrelatlonshipe 
between  the  varioua  components  of  training? 


1. 

not  at  all 

2. 

very  little 

3. 

somewhat 

4. 

to  a  fair  extent 

5. 

very  much 

To  what  extent  were  the  benefita  of  the  SADT  project  at  TRADOC,  if 
any,  due  to  the  diagraaa  or  models  themselves,  i.e, ,  the  tangible 
products  or  drawings,  as  opposed  to  the  process  of  communication 
between  ODCST  staff  members  which  may  have  been  stimulated  by  the 
modeling  process?  If  you  feel  no  benefits  accrued  at  all  relevant 
to  this  question,  check  only  the  following: 

_  no  benefits  at  all 


1. 

benefits 

2. 

benefits 

3. 

benefits 

produced 

4. 

benefits 

5. 

benefits 

were  entirely  product -diagram  related 

were  mostly  product-diagram  related 

were  split  about  evenly  between  products 
and  processes  stimulated 

were  mostly  process  related 

were  entirely  process  related 


To  what  extent  has  the  application  of  SADT  at  TRADOC  saved  time 
relative  to  other  approaches  (e.g. ,  in-house  TRADOC  committees, 
outside  consultants)  that  might  have  been  employed  to  develop  an 
integrated  system  concept  of  training? 


1. 

has 

not  saved  time 

2. 

has 

saved  very  little  time 

3. 

has 

saved  some  time 

4. 

has 

saved  a  fair  amount  of  time 

5. 

has 

saved  considerable  time 
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7.  To  what  extent  did  the  Task  3  eodel  of  Army  training  provide  a 
clearer  understanding  of  the  Amy's  training,  testing,  and  evalu¬ 
ation  programs? 

_ ___  unfamiliar  with  Task  3  model 


1 

l  . 

1. 

less  understanding  than  before  project 

2. 

same  understanding  as  before  project 

4  . 

3. 

a  slightly  better  understanding 

i 

1 

J 

4. 

a  noticeably  better  understanding 

5. 

a  much  better  understanding 

f 

4  . 

s 

8. 

Do  you  believe  the  SADT  models  produced  by  the  SofTech  project  will 
actually  be  used  in  planning  needed  changes  in  Army  training? 

1. 

will  not  be  used 

! 

2. 

will  be  used  to  a  small  extent 

3. 

will  be  used  some 

! 

*  - 

4. 

will  be  used  to  a  fair  extent 

m  — 

i 

m  t» 

i. 

5. 

will  be  used  extensively 

9. 

Has  SADT  helped  identify  changes  in  training  procedures  that  would 
have  a  significant  effect  on  combat  readiness  and  battlefield 
effectiveness  if  instituted? 

m  » 

| 

1. 

definitely  not 

a  • 

2. 

a  little 

1. 

3. 

somewhat 

f* 

4. 

to  a  fair  extent 

L 

5.  _ 

to  a  significant  extent 

£ 

K 

I 

1 
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10.  In  your  opinion,  would  the  complete  SADT  model  of  training  serve 
as  a  valuable  future  reference  for  Army  personnel  responsible  for 
providing  training? 

1.  _  would  not  be  of  any  value 

2.  _  would  be  of  little  value 

3.  _ would  be  of  some  value 

4.  _  would  be  fairly  valuable 

5.  _  would  be  very  valuable 

11.  In  your  opinion,  is  SADT  an  effective  approach  to  the  design  and 
analysis  problems  encountered  by  an  agency  such  as  TRADOC? 

1.  _  not  effective 

2.  _  rarely  effective 

3.  _  marginally  effective 

4.  _  fairly  effective 

5.  _  very  effective 

12.  To  what  extent  do  you  believe  SADT  models  will  provide  the  impetus 
for  changes  in  Army  training  and  testing  programs? 

1.  _  will  not  provide  any  impetus  for  change 

2.  _  will  contribute  to  change  to  a  small  extent 

3.  _  will  provide  some  impetus  for  change 

4.  _  will  contribute  to  change  to  a  fairly  large  extent 

5.  _  will  provide  the  major  impetus  for  change 
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13.  To  what  extent  were  the  SADT  models  useful  In  Identifying  existing 
organizational  inefficiencies  In  the  current  training  and  testing 
procedures? 

1.  _  no  use 

2.  _ not  ouch  use 

3.  _ some  use 

4.  _  a  fair  amount  of  use 

5.  _  a  great  use 

14.  Has  the  application  of  SADT  led  to  conceptual  insights  about 
alternative  methods  of  conducting  Army  training  and  testing 
programs? 

1.  _ not  at  all 

2.  _ very  little 

3 .  _ some 

4.  _  to  a  fair  extent 

5.  _  to  a  great  extent 

15.  In  your  opinion,  will  the  SofTech  project  ultimately  affect  the 
training  concepts  held  by  TRADOC? 

1.  _ not  at  all 

2.  _ very  little 

3.  _  to  some  extent 

4.  _  to  a  considerable  extent 

5.  _  to  a  great  extent 
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16.  In  your  opinion,  would  it  be  useful  to  TRADOC's  mission  to  trein 
additional  personnel  to  be  SADT  authors? 


1. 

no  use 

2. 

slightly  useful 

3. 

of  some  use 

4. 

fairly  useful 

5. 

very  useful 

17.  Beyond  the  specific  models  produced,  to  what  extent  has  the 

experience  of  participating  in  this  project  increased  TRADOC's 
general  analytic  and  planning  capabilities? 

1.  _  the  experience  has  had  no  effect  on  these  abilities 

2.  _  the  experience  has  had  a  minimal  effect  on  these 

abilities 

3.  _  the  experience  has  had  an  effect  only  in  the 

particular  content  areas  modeled 

4.  _  the  experience  has  had  a  noticeable  effect  on  these 

abilities 

5.  _  the  experience  has  greatly  increased  these  abilities 

within  TRADOC 


18.  In  your  opinion,  is  it  realistic  to  expect  that  TRADOC's  general 
analysis  and  planning  capabilities  would  increase  as  a  result  of 
their  participation  in  a  project  such  as  this  one? 

1.  _  not  a  realistic  expectation 

2.  _  definitely  a  realistic  expectation 

3.  _  insufficient  information  to  judge 


146 


19.  In  your  opinion,  has  the  amount  of  progress  made  by  this  project 
on  the  problems  facing  TRADOC  been  worth  the  time  Army  personnel 
have  devoted  to  it? 

1.  _  definitely  has  not  been  worth  the  time 

2.  _  generally  has  not  been  worth  the  time 

3.  _  has  varied  with  the  content  being  modeled 

4.  _  generally  has  been  worth  the  time 

5.  _  all  time  spent  has  been  worthwhile 


20.  The  new  total  system  conceptualization  of  training  developed  by 
TRADOC  is  the  result  of  several  major  intellectual  activities 
occurring  at  TRADOC  in  the  last  few  years.  We  are  trying  to 
determine  how  much  influence  the  SofTech  project  has  had  on  the 
emergence  of  this  new  system.  Please  Indicate  your  estimate 
of  SofTech*  s  contribution  to  the  progress  made  by  TRADOC  in 
developing  an  integrated  systems  approach  to  training. 

1.  _  SofTech  has  not  contributed 

2.  _  SofTech' 8  contribution  has  been  minimal 

3.  _  SofTech  made  a  moderate  contribution 

4.  _  SofTech  was  a  major  contributor 

5.  _  SofTech  was  the  most  important  single  contributor 


21.  Do  you  consider  the  total  system  concept  of  training  as  elucidated 
by  the  Task  3  model  an  improvement  over  previous  conceptualizations 
of  training  held  by  the  Army? 

1.  _  previous  conceptualizations  of  training  were 

superior 

2.  _  previous  conceptualizations  were  slightly  better 

3.  _  previous  conceptualizations  and  total  system  concept 

are  about  equal 

4.  _  total  system  concept  is  slightly  better 

5.  _  total  system  concept  is  superior 


147 


Please  respond  to  the  following  questions  by  writing  your  answers  In  the 
space  provided.  If  you  need  wore  specs,  continue  on  the  extra  pages 
at  the  end  of  this  booklet  or  attach  an  additional  sheet.  While  sobs 
of  the  questions  could  be  responded  to  with  a  brief  consisnt,  I  would 
like  to  ask  that  you  elaborate  on  your  responses  whenever  possible. 


22.  Questionnaire  and  interview  data  showed  that  the  opinions  of  TRADOC 
officers  were  Bore  positive  towards  SADT  at  the  end  of  Task  3 
(May,  1977)  than  they  wars  at  the  end  of  Task  2  (Deceaber,  1976). 

To  what  do  you  attribute  this  shift  in  attitude? 


IJ 

u 


D 

E 

1 

i 

B 

\ 

j 


To  your  knowledge.  In  whet  weye  will  the  Tank  3  aodel  be  used  by 
TRADOC? 


24.  It  vu  anticipated  that  in  addition  to  tii*.  SAD?  models  produced,  the 
TRADOC  organisation  would  improve  their  analytic  and  planning  capa¬ 
bilities  aa  a  result  of  their  participation  in  the  project. 

In  you  opinion,  has  this  project  had  aa  Impact  on  TRADOC' s  ability 
to  solve  large  and/or  coaplex  problems?  If  so,  how  has  TRADOC 'a 
analysis  and  planning  capabilities  improved?  If  you  do  not  think 
the  project  has  had  an  Impact  on  these  abilities  within  TRADOC,  why 
do  you  think  this  transfer  of  technical  skills  failed  to  occur? 


27.  Do  you  have  any  other  thought a  or  consents  about  SADT  or  the  SofTech 
project  which  you  have  not  had  an  opportunity  to  expreaa  but  believe 
to  be  important  to  thia  evaluation? 
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